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Executive Summary 


The No Child Left Behind Act (NCLB) aims to strengthen state Title I accountability 
systems to spur the improvement needed to reach the law’s goals: that all students will reach 
proficiency by 2013-14. NCLB requires states and districts to report and act on student test 
results in a host of new ways. Among the many actions states and districts must take under 
NCLB’s accountability system are identifying schools that repeatedly do not make progress, 
providing assistance to schools that need to improve, and offering more choices to parents. 


To understand how states, districts, and schools are implementing these accountability 
provisions, the U.S. Department of Education commissioned the Evaluation of Title I 
Accountability Systems and School Improvement Efforts (TASSIE) beginning with the 2001-02 
school year as a baseline year and continuing for two more years. The study has gathered data 
from educators in schools, district offices, and state education agencies through surveys and 
interviews and from parents through focus groups. This report covers the 2002-03 school year, 
and focuses on how the following key aspects of Title I accountability provisions in NCLB were 
carried out during the first year NCLB was in effect: the assistance and interventions provided to 
Title I schools identified for improvement, the steps these schools are taking to improve, and 
public school choice and supplemental services offered to students in these schools. 


States were not required to have everything in place during 2002-03, but they were 
expected to implement many key accountability provisions and meet certain benchmarks for 
others. Findings from 2001-02 suggested that states, districts, and schools were well positioned 
to meet the requirements of NCLB but would need to make substantial changes to fully meet the 
challenges of the new law. Findings from 2002-03 show that states and districts were making 
progress in implementing accountability systems under NCLB, but big gaps remained between 
their existing systems of accountability and the vision embodied in NCLB of coherent systems 
that support all schools and all students to reach high standards. 


Context for the 2002-03 School Year 


The consequences outlined in NCLB for identified schools and districts were to be applied 
during the law’s first full year, the 2002-03 school year, making identified schools and districts 
in 2002-03 the first to be subject to NLCB accountability requirements. Because states were 
developing adequate yearly progress (AYP) definitions following NCLB criteria during 2002-03, 
these definitions were not applied to identify schools and districts in that year. Instead, NCLB 
laid out transition rules for 2002-03: schools and districts were to be identified based on their 
improvement status as of Jan. 7, 2002—the day before NCLB was enacted. For example, a 
school in its first year of improvement as of Jan. 7, 2002, was to be identified as a school in its 
first year of improvement under NCLB. Similarly, a school in its second year of improvement 
as of Jan. 7, 2002, was to be identified as a school in its second year of improvement under 
NCLB. The exception to these transition rules was that a school in improvement on Jan. 7 that 
had made one year of AYP in 2000-01 and could exit improvement status if it also made AYP 
based on its 2001-02 assessment results. States were also permitted to identify schools for 


improvement in 2002-03 that had missed AYP for the first time based on their 2000-01 
assessment results (and thus were not identified for improvement as of Jan. 7, 2002) and also 
missed AYP based on their 2001-02 assessment results. Similar transition rules applied to 
districts. 


Despite the transition rules, case study data indicate that other issues contributed to the 
challenge of achieving widespread understanding of which schools were identified in 2002-03, 
such as changes in some state assessments, the presence in some states of parallel state 
accountability systems that use criteria different from AYP, anticipated changes to state 
definitions of AYP under NCLB, and ongoing confusion at the local level about school 
identification status. For example, in 30 percent of the study’s Title I schools identified for 
improvement in 2001-02 that continued to be identified in 2002-03, the principal was not aware 
of the school’s status. 


Because of transitional policies that applied for only 2002-03 and new definitions of AYP 
under NCLB to be implemented in 2003-04, inferences about trends in numbers of schools and 
districts identified for improvement should be made with caution. In addition, because the 
incentives underlying NCLB cannot be expected to have their full effect if schools and districts 
do not know which schools are identified for improvement, in light of evidence of some 
confusion at the local level about which schools were identified, data from 2002-03 should be 
considered only a first glimpse into the influence of NCLB. The report for the final year of this 
study, 2003-04, will show how implementation of NCLB is changing during its first few years. 


Title | Schools and Districts Identified for Improvement 


Under NCLB, schools that do not make AYP for two consecutive years must be “identified 
for improvement.” The same rule applies to districts. 


In 2002-03, 6,000 schools were identified for improvement under Title I, representing 
13 percent of all Title I schools nationwide. About 9,000 schools were identified in 2001-02. 
Less than half (46 percent) of the Title I schools that had been identified for improvement in 
2001-02 continued to be identified in 2002-03. Schools that remained in improvement status did 
not differ on factors such as district and school size and poverty level, and school level 
(elementary, middle or high school) from schools that were no longer identified. About one- 
third (approximately 2,000) of the identified schools in 2002-03 were newly identified. 


A similar pattern held for districts identified for improvement. In 2002-03, eight states 
reported they had identified any of their districts for improvement, and the number of districts 
identified for improvement decreased from 15 percent in 2001-02 to 5 percent in 2002-03. 


Of roughly 11,200 districts nationwide with Title I schools, 16 percent (around 1,900) 
had at least one identified Title I school in 2002-03 compared with 26 percent (around 
2,900) in 2001-02—a decline of 10 percent.’ Of the Title I schools identified in 2002-03, a 


' The district sample is a nationally representative sample of all districts receiving Title I funds. Percentages can 


vary from estimated numbers due to rounding of estimates. These estimates are based on data from the TASSIE 
Eligibility Dataset and vary slightly from estimates derived from respondents to the district survey due to 
rounding (see page xviii). 


higher percentage were located in urban and high-poverty districts than in rural and lower- 
poverty districts (see Exhibit S1), but almost half (47 percent) were still located in districts with 
enrollments under 10,449 students (52 percent in 2001-02). 


Exhibit $1 


Title | Schools Identified for Improvement and All Title | Schools in 2002-03, 
by District Size, Location and Poverty Distributions 


Number of Percent of 
Title | schools Percent of all schools 
identified for Title | identified Percent of all identified within 
improvement schools Title | schools categories 
District size, by student 
enrollment 
Small (200 to 3,503) 1,738 29 40 9 
Medium (3,504 to 10, 448) 883 15 24 8 
Large (10, 449 to 37,740) 1,246 21 17 15 
Very large (>37, 740) 2,135 36 19 24 
Total 6,002 100 100 13 
Location 
Rural 1,766 29 42 9 
Suburban 1,416 24 32 9 
Urban 2,820 47 25 23 
Total 6,002 100 100 13 
District poverty, by percent of 
children living in poverty 
Lowest poverty (<11 percent) 701 12 23 6 
Middle poverty (11 to 22 percent) 1,682 28 36 10 
Highest poverty (>22 percent) 3,618 60 41 18 
Total 6,002 100 100 13 


Exhibit reads: An estimated 1,738 Title I schools in small districts were identified for improvement in 2002-03, 
which represent 29 percent of all identified Title I schools in the nation. Small districts contained an estimated 
19,188 Title I schools overall, or 40 percent of all Title I schools. In small districts, an estimated 9 percent of 
Title I schools (i.e., 1,738 divided by 19,188) had been identified for improvement. 


Most districts with identified Title I schools had very few identified schools overall. 
Across all districts with identified Title I schools, the majority (57 percent) had only one school 
identified and tended to be small districts with few schools (see Exhibit S2). In terms of 
proportions, 39 percent of districts with identified schools had less than a quarter of their schools 
identified for improvement and 12 percent had over three-quarters of their schools identified. 
Smaller districts were more likely than others to have high proportions of identified schools 
because their total number of schools is small. 
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Exhibit $2 


Distribution of Identified Title | Schools Across All Districts 
and Districts With Identified Title | Schools 


Number of schools Percent of districts 
identified for improvement | Percent of all districts {with identified schools 
0 84 NA 
1 10 57 
2 3 15 
3-4 2 13 
5-12 2 10 
13+ <1 2 


Exhibit reads: Eighty-four percent of all districts had no Title I schools identified for 
improvement in 2002-03. Fifty-seven percent of districts with identified schools had only 
one identified school. 


School Improvement and District Assistance in Identified Schools 


NCLB requires states and districts to have systems of support in place for Title I schools 
identified for improvement. States must establish statewide systems of school support and 
districts must provide ongoing technical assistance to schools as they develop and implement 
their school plans. In particular, districts must help schools analyze student achievement data 
and develop plans for improvement, revise their budgets so that resources are effectively 
allocated to the activities most likely to increase student academic achievement, implement 
professional development, and put in place instructional practices that have shown evidence of 
effectiveness. 


Although most states and districts provided school support teams or distinguished 
educators to identified schools in 2002-03, a third of states had not done so, and many 
identified schools reported that they had not received either of these forms of assistance.” 
Twenty-eight states organized school support teams, distinguished educators, or both to provide 
assistance to identified schools. Similarly, nearly two-thirds (61 percent) of districts provided 
support teams, principal coaches or mentors, or distinguished educators to identified schools. 
This assistance ranged from occasional visits to full-time presence in the school.” 


However, nearly two-thirds (62 percent) of principals of schools that continued to be 
identified in 2002-03 reported that their schools had not received help from a school support 
team, a principal coach or mentor, or a distinguished teacher (see Exhibit S3). The proportion of 
schools receiving at least one of these forms of assistance had not improved since the year 
before. In fact, assistance from school support teams was received by fewer schools in 2002-03 


N 


Identified schools in this section refer to the sample of Title I schools that were identified for improvement in both 
2001-02 and 2002-03. In 2001-02, the sample of identified schools was nationally representative. In 2002-03, the 
sample is the subset that were continuously identified in both years. 

Descriptive statements without reference to statistics are based on case studies of 20 schools and their 15 districts. 
These data were used to enhance understanding of survey results. 
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(28 percent) than the previous year (46 percent). This drop is surprising, since these schools had 
been identified for one additional year, and therefore should have been in even greater need of 
assistance. 


Exhibit S3 


School Reports of On-site Assistance, Among 
Title | Schools That Continued to Be Identified for Improvement 


Percent of identified schools reporting 
that they received assistance 


School support team 25 
Principal mentor or coach 20 
Distinguished teacher 13 


Any of these forms of assistance (i.e., school support 


team, principal mentor, or distinguished teacher) a8 


Exhibit reads: Twenty-five percent of all Title I schools that continued to be identified for improvement in 2002-03 
reported that they received help from a school support team. 


Even when continuously identified schools were located in districts or states that provided 
some form of on-site assistance, the proportion reporting that they received that assistance was 
still low. Among schools in districts that fielded school support teams, only a third reported that 
they received assistance from a team. Similarly, only a third of the continuously identified 
schools located in states that fielded support teams reported that they received assistance from a 
team. Although 91 percent of schools that continued to be identified for improvement in 2002- 
03 were located in districts or in states that reported offering some type of on-site assistance to 
identified schools (either school support teams, distinguished teachers, or other types of 
assistance), the evidence suggests that these forms of on-site assistance were not available to 
every school. 


Finally, 9 percent of schools that continued to be identified in 2002-03 were located in 
districts where neither the state nor the district offered school support teams, distinguished 
teachers, or principal mentors. 


Not all districts and schools had the organizational capacity in place—including 
staffing, time, and materials—that would enable them to provide and take advantage of 
assistance; this was particularly true for small and rural districts. Small and rural 
districts—which served roughly a third of all identified schools in 2002-03—had limited staff 
and resources to provide assistance to identified schools. Only half (49 percent) of rural districts 
provided school support teams, distinguished teachers, principal mentors, or some combination 
of the three, to their identified schools, compared with three-quarters of urban districts (75 
percent). In contrast, large and urban districts were more likely to provide assistance of various 
kinds. For example, only one-fifth (21 percent) of small districts provided additional full-time 
staff to support teacher development in identified schools whereas two-thirds (64 percent) of 
very large districts provided such assistance (see Exhibit S4). 
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Exhibit S4 
Assistance Provided by the District to Identified Schools, by District Size 


100 5 
90 4 86 


80 4 77 a4 78 


63 64 


60 4 57 
52 53 


50 
47 


43 45 45 


25 27 24 
21 
20 | 18 
10 5 6 
0 


Percent of districts with identified schools 
a 
oO 


School support team Distinguished teacher Principal mentor or Any of these forms of Morethanone FTE — Additional full-time 


coach assistance (i.e., staff member per school-based staff to 
school support team, identified school support teacher 
District size distinguished teacher, | employed by the development 
or principal mentor) district to provide 
OSmall OMedium mLarge OVery Large assistance 


Exhibit Reads: Among districts with identified schools, 43 percent of small districts, 57 percent of medium 
sized districts, 67 percent of large districts, and 77 percent of very large districts provided school support teams 
to identified schools in 2002-03. 


Schools also need a minimal organizational structure in place—including adequate 
staffing—to be able to benefit from assistance that is provided. In case study districts, identified 
schools with minimal instructional leadership, limited resources, and high concentrations of poor 
children struggled to benefit from district assistance. 


School improvement efforts and district assistance focused predominantly on 
increasing the use of achievement data, matching curriculum and instruction with standards 
and assessments, and undertaking school planning. In their efforts to improve low-performing 
schools, 86 percent of districts with identified schools reported placing a major focus on aligning 
curriculum and instruction with standards and assessments and 77 percent reported a major focus 


on increasing the use of student achievement data to inform instruction and school improvement 
(see Exhibit S5). 
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Exhibit S5 


Percent of Districts That Placed a Major Focus on Strategies for Improving 
Low-Performing Schools, Among Districts With Identified Schools 


100 

90 + 86 

80 + 77 

70 + 

63 

2 61 
2 60+ 
2 
To 
‘oo 50+ 
€ 
8 
o 405 37 
oO 

30 28 

20 4 

10 5 

0 T T 
Curriculum Use of data School planning Professional Newcurriculum or School reform 
alignment process development instructional model 
program 
School improvement strategy 


Exhibit reads: Among districts with identified schools, 86 percent reported that matching curriculum and 
instruction with standards, assessments, or both is a major focus of district resources for low-performing 
schools. 


The percentage of continuously identified schools citing the use of student achievement 
data as a major focus area for school improvement increased from 75 percent in 2001-02 to 86 
percent in 2002-03. However, school uses of achievement data focused more on schoolwide 
planning and less on instructional decisions about students. In the case study districts, whether 
or not schools used data for instructional decisions about students was related to the availability 
of diagnostic data and assistance in determining its implications for instruction. Even uses of 
data in planning were limited; for example, although almost all (93 percent) continuously 
identified schools developed school plans, only about half (53 percent) monitored progress 
toward the plan’s goals at least quarterly. 


Curriculum alignment with standards and assessments was reported by districts as a major 
focus of school improvement; however, integrating new curricula with existing practices was not 
a focus of district or school improvement efforts. Almost one-third (32 percent) of continuously 
identified schools adopted new curricula in both language arts and math in the last three years 
(i.e., between 2000-01 and 2002-03). In case study districts, schools layered new 
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programs on top of existing programs, raising issues of how to cope with multiple new programs 
and how best to mesh the new with the old. 


Of the continuously identified schools that adopted new reading or mathematics 
curricula or school reform models, most reported emphasizing related professional 
development or receiving a minimum level of related district assistance; fewer reported 
receiving both.’ For example, of the almost six in ten schools (58 percent) that continued to be 
identified and reported adopting a new language arts program in the last three years, 85 percent 
reported either emphasizing related professional development or receiving a minimum level of 
related district assistance (see Exhibit S6). At the same time, less than half (43 percent) reported 
receiving both. Similarly, almost three-quarters (71 percent) of schools adopting a new 
mathematics curriculum (48 percent) reported either emphasizing related professional 
development or receiving a minimum level of related district assistance while 11 percent reported 
receiving both. 


Districts reported providing professional development to identified schools at higher 
rates than schools with adopted improvement strategies reported receiving such support. 
Among districts with identified schools, 95 percent reported supporting professional development 
in language arts instruction in identified schools. One would expect the highest rates of 
professional development for schools adopting new curricula yet only 72 percent of such schools 
reported emphasizing related professional development. This represented 42 percent of all 
identified schools. A similar pattern held for mathematics. Among districts with identified 
schools, 80 percent reported supporting professional development in mathematics instruction in 
identified schools, yet only 38 percent of continuously identified schools that had adopted a new 
mathematics curriculum reported emphasizing professional development. This represented 18 
percent of all continuously identified schools. These differences might be due to the amount (for 
example, only one workshop) and quality of the professional development provided by districts, 
and whether or not it met the needs of the teachers. If districts provided only minimal or poor 
quality training, schools were unlikely to describe it as an area of emphasis. Another reason for 
the difference in responses may be that districts supported professional development in some 
identified schools but not in others. 


Additional school staff without classroom responsibilities (e.g., staff developers, 
curriculum specialists) played an important role in school improvement in the case studies. Yet 
only a third (33 percent) of districts with identified schools reported that identified schools had 
been assigned additional full-time staff whose specific job was to support teacher development. 
Even fewer (23 percent) continuously identified schools reported that they had been assigned 
such staff by their district or state. 


Although districts provided a range of assistance to identified schools, they seldom targeted 
assistance and interventions only to schools identified for improvement. Districts typically 
provided some kinds of assistance to all of their schools, whether or not they were low- 
performing or identified for improvement under Title I. For example, nearly three-quarters (71 


* The definitions and scales underlying “emphasizing related professional development” and “receiving a minimum 
level of related district assistance” are described in the appendix. 
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Exhibit S6 


Of Continuously Identified Schools Engaged in Different Improvement Activities, 
Percent That Emphasized Related Professional Development or Received 
a Minimum Level of Related District Assistance 


90 85 [| Of schools engaged in activity, 
80 - percent that emphasized related 
f2 71 professional development OR 
70 - received a minimum level of 
59 related district assitance 
» 604 a 
8 50 - 47 Of schools engaged in activity, 
9 43 percent that emphasized related 
> 40-4 38 38 professional development 
5 
Q 30 | Of schools engaged in activity, 
2 20 | 17 percent that received a minimum 
11 level of related district assistance 
10 + 5 
0 Of schools engaged in activity, 
0 percent that emphasized related 
‘ professional development AND 
New New mathematics School reform racsived amminimurilevel or 
reading/language curriculum model related district assistance 
arts curriculum 
School improvement activity 


Exhibit reads: Among schools that continued to be identified for improvement in 2002-03 and reported 
implementing a new language arts program, 85 percent reported emphasizing related professional development 
or receiving a minimum level of related district assistance; 72 percent reported emphasizing related 
professional development; 59 percent reported receiving a minimum level of related district assistance; and 43 
percent reported emphasizing related professional development and receiving a minimum level of related 
district assistance. 


Note: See the appendix for sample sizes and additional statistical information. 


percent) of districts helped all their schools write a school improvement plan, including those 
that were higher-performing. Only a small number (13 percent) provided such assistance only to 
identified schools. Similar ratios held for other types of support. In addition, almost all districts 
had the same reading and mathematics curriculum requirements for all schools, whether they 
were higher-performing, low-performing, or identified for improvement under Title I. This may 
explain why principals of continuously identified schools cited reasons of instructional 
consistency (i.e., standardized curricula), not Title I identification, for adopting new reading and 
language arts curricula. 


Although most districts provided the same kinds of curricula and assistance to all their 
schools, some attempted to provide additional resources to their low-performing schools through 
targeting external grants or more intensive assistance. However, districts rarely distinguished 
between schools identified for improvement and other low-performing schools because, as 
observed in case study districts, schools move in and out of identification status and were 
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perceived to have similar needs. This helps explain the finding that districts with low- 
performing schools, none of which were identified for improvement, provided similar kinds of 
support as districts with identified schools. 


Public School Choice and Supplemental Services Under Title | 


NCLB required districts with identified Title I schools to implement public school choice 
under Title I and supplemental services as of the beginning of the 2002-03 school year. All 
districts with any Title I school identified for improvement (16 percent of all Title I districts 
nationwide or 1,800 school districts)’ were required to provide parents of students attending 
identified Title I schools the choice to transfer their child to another public school in the district 
not identified for improvement and provide transportation (see Exhibit S7). Districts with Title I 
schools in their second year of improvement, corrective action or restructuring (11 percent of 


Exhibit S7 


Participation in Title | Choice and Supplemental Services in 2002-03, 
Among Districts With Identified Schools 


School choice Supplemental services 
Districts: 
Number required to offer option 1,800 1,100 
Number where option offered 1,200 500 
Percent where option offered 67% 48% 
Schools: 
Number where option required 6,000 1,300 
Number where option offered 5,100 800 
Percent where option offered 84% 58% 
Students (among districts that 
provided options): 
Number eligible 1,535,000 592,000 
Number that participated 18,000 42,000 
Percent that participated 1% 7% 


Exhibit reads: Among districts with identified Title I schools, an estimated 1,800 were required to offer school 
choice to students enrolled in identified Title I schools and 1,100 districts were required to offer supplemental 
services to low-income students in Title I schools identified for two or more years. The data presented in this 

exhibit are estimates at the 95 percent confidence interval with 32 degrees of freedom and estimates have been 
rounded. 


Note: The number of students eligible is underestimated because not all districts that should have provided 
choice and supplemental services reported providing these options and did not provide eligibility data. See 
Exhibits 21, 30 and 31 for explanations regarding the limitations of these data. 


> The 1,800 estimate was derived from respondents to the district survey which represented 16 percent of all Title I 


districts when rounded. Survey analyses were based on the 1,800 districts with identified schools in 2002-03 (see 
the appendix regarding statistical analyses). 
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districts nationwide or 1,100 LEAs) were required to offer supplemental services (e.g., tutoring) 
in addition to school choice. Because Title I school choice and supplemental services were new 
requirements for most districts in 2002-03 (though some more limited Title I choice 
requirements applied to certain schools in 1999-2000), implementation for most districts in 
2002-03 represented their initial efforts to implement these requirements. 


The majority of districts required to offer school choice to parents of students 
attending a Title I identified school did so in 2002-03. Two-thirds (67 percent) of districts 
with identified Title I schools (about 1,200 districts) offered public school choice and 62 percent 
of districts with identified schools offered all non-identified schools in the district as an option to 
parents of students in Title I identified schools (see Exhibit S8). Districts that provided Title I 
choice included approximately 5,100 identified schools or 84 percent of the identified Title I 
schools in 2002-03. Districts that did not offer public school choice (33 percent of those 
required to do so) were predominantly small, rural, and poor. 


Exhibit S8 


School Choice Options Available to Parents of Children Enrolled in 
Identified Title | Schools, Among Districts That Provided Choice 


School choice options provided Bakagls of metiets 
with identified schools 

All other schools in the district at the appropriate grade 62 

level that are not identified for improvement under Title | 

A subset of schools that have been paired with the sending 19 

school 

All other schools within a certain geographic zone 15 

Public schools outside the district 5 

Other 8 


Exhibit reads: Among all districts with identified schools that offered Title I choice, 62 percent 
offered all other schools in the districts at the appropriate grade level that were not identified for 
improvement as a choice option. 


In 2002-03, few parents exercised their option to transfer to a non-identified school. 
In the 67 percent of districts that offered choice under Title I, there were an estimated 1.5 million 
students eligible to exercise choice because they attended an identified Title I school. About 2 
percent (28,000 students) requested a transfer to a non-identified school and about | percent 
(18,000 students) then transferred to another school. Over three-quarters (77 percent) of districts 
with identified schools that offered choice had nine or fewer students transfer out of identified 
schools. Eighteen percent had between 10 and 99 students and 5 percent had 100 or more 
students transfer out of identified schools. 


As was true in 2001-02, districts pointed to lack of space and an inability to create 
additional space as the greatest challenges to successful implementation of public school choice 
in 2002-03. NCLB does not exempt districts facing these obstacles from the requirement to 
offer Title I choice, although districts may take capacity into account when deciding which 
schools to offer as transfer options. Small districts were more likely than others to have 
responded that no alternate schools existed or that transportation was a serious problem, whereas 
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suburban districts were more likely than others to have responded that lack of space in alternate 
schools was a serious problem. Given the smaller number of middle and high schools compared 
to elementary schools in a district, choices for students in the upper grades were far more limited 
in all districts and essentially nonexistent in small and rural districts. 


About half of the districts required to offer supplemental services in 2002-03 did so; 
in those districts, few students received services. Forty-eight percent of districts (an estimated 
500 LEAs) required to offer supplemental services provided these services; 62 percent of 
districts required to offer supplemental services made the services available to all low-income 
students because demand did not exceed funding (i.e., the amount equal to 20 percent of the 
district Title I, Part A, allocation). As was true with choice, districts that did not offer 
supplemental services were predominantly small, rural, and poor. Of the 592,000 eligible 
students in districts that offered supplemental services in approximately 800 identified Title I 
schools, roughly 7 percent (42,000 students) received services from an approved provider in 
2002-03. 


Approved providers in 2002-03 represented a variety of organizations, but private 
providers (e.g., not-for-profit private groups or for-profit organizations) made up the largest 
percentage (70 percent), many of which were national providers. Although the numbers of 
providers in states continued to increase in 2003, they did not adequately address some local 
needs: providers were scarce in rural areas, some did not serve districts with small numbers of 
students for financial reasons, and some did not serve students with special needs. These were 
among the challenges districts faced in ensuring the provision of supplemental services in 2002- 
03 (see Exhibit S9). 


Parents faced challenges in understanding their options and in acting on them. Many 
districts (55 percent) with identified Title I schools that provided choice missed the deadline to 
notify parents of students in these schools of their public school choice option before the 
beginning of the 2002-03 school year. One-fifth (21 percent) of districts with eligible students 
had not communicated with parents as of spring 2003 regarding their right to receive 
supplemental services because the district had not yet begun to provide those services. Delays in 
identifying which schools were identified for improvement in 2002-03 and the fact that very few 
states (only five) had given a list of approved providers to districts before the beginning of the 
2002-03 school year contributed to the delays in notifying parents. Even when parents received 
notification materials, they did not necessarily understand the information contained in the 
materials and some districts did not clearly communicate the options open to them. 


Nevertheless, according to focus groups with parents in identified schools, awareness of 
options appeared to be only one factor that influenced parents’ choices. Few parents requested 
supplemental services for reasons that included preexisting after-school programs or providers 


° Only 55 percent of the 11 percent of districts required to offer supplemental services in 2002-03 reported that they 
had students eligible to receive supplemental services and, hence, the number of students eligible for 
supplemental services is underestimated (but the number of students receiving supplemental services is not since 
these data are based on districts actually providing services). 
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Exhibit S9 


Challenges Faced by Districts That Implemented Supplemental Services, 
Among Districts That Provided Services 


Great Moderate Small Not at 

Type of challenge extent extent extent all 
Lack of providers in the area 48 6 6 A1 
Lack of an established reputation for providers 18 27 24 30 
with parents 
Competition from existing after-school programs 16 8 33 44 
Inadequacy of an amount equal to 20% of the 
district’s Title | allocation for meeting all requests 
for services or providing transportation to all 6 3 11 81 
students who request a transfer to alternate 
schools 
Inadequate time for parents to learn about 3 13 22 62 
supplemental services 
Inadequate information for parents about 

: 1 8 20 71 

supplemental services 


Exhibit reads: Among districts required to offer and provided supplemental services in 2002-03, 48 percent 
responded that a lack of providers in the area was a challenge to a great extent to implement supplemental 
services, 6 percent responded that such a lack was a challenge to a moderate extent, 6 percent responded that 
it was a challenge to a small extent, and 41 percent responded that it was not a challenge. 


located far from the school. Similarly, the small percentage of students that requested a transfer 
to a non-identified school were associated with several factors parents considered other than the 
school’s identification for improvement. Parent focus groups revealed that parents were 
concerned about the performance of their children rather than the school as a whole and 
considered factors such as the availability of special programs and services, qualities of 
individual staff members, proximity of the school, and a desire to have their children with others 
from the neighborhood. 


Corrective Actions 


NCLB requires districts to take a set of corrective actions with schools that do not make 
AYP for two years after being identified for improvement (“corrective action” schools). These 
actions range from those that overlap with assistance and support (e.g., requiring a new research- 
based curriculum) to those that are clearly interventions (e.g., replacing school staff). 


Few districts in 2002-03 had identified schools subject to corrective actions; therefore, the 
impacts of NCLB’s corrective actions were not broad. Only about 400 districts (4 percent) 
nationwide reported that they had schools identified for improvement for three years or more; 
most focused on assistance (i.e., requiring the implementation of a new research-based 
curriculum. 
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Conclusions 


During the first year NCLB was in effect, states, most districts, and schools took actions 
consistent with NCLB requirements, but the gap between the expectations of the law and 
practice remained large in 2002-03: not all NCLB reporting requirements had been implemented, 
two-thirds of identified schools reported no access to certain key forms of state or district on-site 
assistance, school improvement efforts were often not accompanied by related professional 
development or other supports, a third of districts with identified schools did not provide public 
school choice, and half of districts with schools identified for two or more years did not offer 
supplemental services. States and districts with more established organizational structures for 
supporting school improvement efforts and the staff to carry them out were in a better position to 
implement NCLB’s requirements than those with little or no such systems in place. Yet even in 
these states and districts with established structures to support school improvement, aligning 
existing practices with NCLB was proving to take time. Given the complexity of the NCLB 
legislation, findings based on data from the first year of implementation should be interpreted 
with caution. The full report provides more detailed 2002-03 data from both the surveys and the 
case studies. The next and final report based on this study, which will focus on the 2003-04 
school year, will build on findings from 2002-03 to show how NCLB implementation is 
changing over time. 
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| Introduction 


The No Child Left Behind Act (NCLB) aims to strengthen state Title I accountability 
systems to spur the improvement needed to reach the law’s goals: that all students will reach 
proficiency by 2013-14. NCLB, the 2001 reauthorization of the Elementary and Secondary 
Education Act (ESEA), requires states and districts to report and act on student test results in a 
host of new ways. Among the many actions states and districts must take under NCLB’s 
accountability system are identifying schools that repeatedly do not make progress, providing 
assistance to schools that need to improve, and offering more choices to parents. 


To understand how states, districts, and schools are implementing these accountability 
provisions, the U. S. Department of Education commissioned the Evaluation of Title I 
Accountability Systems and School Improvement Efforts (TASSIE), beginning with 2001-02 as a 
baseline year and continuing for two more years. Findings from the 2001-02 school year 
suggested states, districts, and schools were well positioned to meet the requirements of NCLB 
but would need to make substantial changes to fully meet the challenges of the new law. This 
report covers the 2002-03 school year. 


The 2002-03 school year was the first year that states and districts that received Title I 
funds had to respond to the new accountability requirements in NCLB. States were not required 
to have everything in place during 2002-03, but they were expected to implement many key 
accountability provisions and meet certain benchmarks for others. 


The consequences outlined in NCLB for identified schools and districts were to be applied 
during the law’s first full year, the 2002-03 school year, making identified schools and districts 
in 2002-03 the first to be subject to NLCB accountability requirements. Schools and districts, 
however, were not identified based on definitions of AYP developed under NCLB. Those 
definitions did not take effect until the 2003-04 school year. Rather, for 2002-03, schools and 
districts maintained the improvement status they held as of Jan. 7, 2002, under the prior 
authorization of ESEA. The 2002-03 data in this report thus offer only a first glimpse into the 
intended influence of NCLB. 


Against this backdrop, districts moved forward with notifying the public about school 
performance, provided assistance to identified schools, offered school choice and supplemental 
educational services to children in Title I schools identified for improvement (“identified 
schools”), and applied interventions to schools that were not sufficiently improving. Similarly, 
schools continued to focus on planning for and implementing improvement efforts. 


The report focuses on the following key issues, tied to the major accountability provisions of 
NCLB: 


¢ The characteristics of schools and districts identified for improvement in 2002-03 
(Chapter I). 


¢ Support provided to identified schools, the school improvement efforts undertaken by 
these schools to increase student achievement, and the ways that states and districts 
communicated to parents and the public about school performance (Chapter II). 


¢ Efforts to offer public school choice and supplemental educational services to eligible 
students in identified Title I schools (Chapter I). 


¢ Interventions for identified schools—the corrective actions and restructuring or 
alternative governance arrangements imposed by districts on schools that continued not 
to make adequate yearly progress in 2002-03 (Chapter IV). 


Background: Accountability Provisions of Title I 


NCLB frames a common goal for educators: to ensure that no child, regardless of 
background, is left behind by the nation’s public education system. In particular, NCLB has 
strengthened accountability requirements for schools, districts, and states, and sought to address 
the shortcomings and uneven implementation of accountability systems under the previous 
reauthorization of Title I. 


NCLB requires states to develop and implement a single, statewide accountability system 
that will be effective in ensuring that all districts and schools make AYP, and to hold those 
schools that do not do so accountable. The legislation was designed to help all students reach 
proficiency in a specified time period (by 2013-14) by requiring that states create annual 
assessments that measure what children know and can do in reading and mathematics in grades 3 
through 8 as well as testing at least once between grades 10 and 12 by 2005-06, and in science 
by 2007-08 (see Exhibit 1).’ Assessment data are to be disaggregated for students by poverty 
level, race, ethnicity, disability, and limited English proficiency (LEP) to ensure that attention is 
focused on all students and that all schools are held accountable for reaching AYP targets for 
students in each major subgroup at the school. To AYP calculations based primarily on state 
assessment results in language arts and mathematics, NCLB adds minimum assessment 
participation rates of 95 percent, graduation rates for high schools, and at least one other state- 
selected academic indicator for elementary and middle schools. Annual school “report cards” 
are required to provide comparative information on the quality of all schools so that parents can 
make more informed choices about their children’s education. The report cards are to show not 
only how well students are doing in regard to meeting standards, but also the progress that 
disaggregated groups are making in closing achievement gaps. 


7 NCLB does not require that science assessments be used to determine AYP. 


Exhibit 1 


Summary of Key NCLB Accountability Requirements 


Topics 


NCLB requirements 


Single, statewide 
system 


All students must be assessed by the same state assessment, and 
AYP definitions must apply to all schools and districts in the state, both 
Title | and non-Title I. 


Public reporting 


State and district report cards are required to include information on 
state assessment results, schools and districts identified for 
improvement, and certain other information. 


Performance measures 
used in AYP definitions 


Annual state assessments must be administered to students in 
reading or language arts and mathematics. Assessments must be 
administered at least once in grades 3-5 and 6-9 until 2005-06, when 
all grades 3-8 must be assessed; assessments also must be 
administered at least once in grades 10-12. Assessment participation 
rates, graduation rates, and another academic indicator for elementary 
and middle schools must be included. States can include additional 
indicators. 


Criteria for state 
definitions of AYP for 
schools 


The criteria must include absolute targets for each measure of school 
performance. Participation rate targets must be at least 95 percent. 


The criteria must provide for all students reaching proficiency within 12 
years, and set annual measurable objectives and intermediate goals. 
The baseline must be calculated following a specific formula. 


The criteria must include separate, absolute targets for key groups of 
students (all, major racial and ethnic groups, economically 
disadvantaged students, students with disabilities, and LEP students). 


Districts may use additional criteria to add schools to, but not subtract 
them from, state lists of identified schools. 


AYP for districts 


The formula for AYP must be the same for districts as for schools. 


Identification of 
schools for 
improvement 


Schools that do not make AYP for two consecutive years must be 
identified for improvement under Title I. 


Identification of 
schools for corrective 
action 


Schools that do not make AYP for four years (after two years in 
improvement status) must be identified for corrective action. 


Identification of 
schools for 
restructuring 


Schools that do not make AYP for five and six years (after three and 
four years in improvement status) must be identified for restructuring 
(planning for restructuring for one year, then restructuring the following 
year). 


Exiting improvement, 
corrective action and 
restructuring status 


A school or district may exit from improvement, corrective action or 
restructuring status when it makes AYP for two consecutive years 
following its identification for improvement, corrective action or 
restructuring. 


Exhibit 1 (Concluded) 


Summary of Key NCLB Accountability Requirements 


Topics 


NCLB requirements 


Identification of 
districts for 
improvement 


Districts that do not make AYP for two consecutive years must be 
identified for improvement under Title I. 


Districts that do not make AYP for four years (after two years of 
improvement status) must be identified for corrective action under 
Title |. 


District assistance for 
schools identified for 
improvement 


Districts must provide technical assistance to identified schools to: 

¢ Develop and implement their school improvement plans. 

¢ Analyze assessment data. 

¢ Identify and implement professional development, instructional 
strategies, and methods of instruction derived from scientifically 
based research and that have proven effective in addressing the 
specific instructional issues that caused the school to be identified. 

¢ Analyze and revise the school budget to allocate more effectively 
allocate school resources to support activities most likely to 
increase student achievement. 


Consequences for 
schools identified for 
improvement 


Schools must develop or revise a school improvement plan. 


Districts must offer all students in identified schools public school 
choice. 


Districts must offer students from low-income families in identified 
schools supplemental educational services from an approved provider 
(beginning in year two of improvement). 


Consequences for 
schools identified for 
corrective action 


Consequences from years one and two of improvement continue. 


Districts must implement one of a series of “corrective actions” defined 
in the legislation. 


Consequences for 
schools identified for 
restructuring 


Districts must spend the first year planning to implement at least one 
of a series of school “restructuring efforts.” During the second year, 
districts must implement the schools’ restructuring plans. 


Consequences for 
districts identified for 
improvement 


Districts must develop or revise a school improvement plan. 


States must provide technical assistance to identified districts. 


Consequences for 
districts identified for 
corrective action 


States must implement one of a series of “corrective actions” for 
identified districts. 


State assistance for 
identified districts and 
schools 


States are required to establish a statewide system of 
support—including school support teams, distinguished principals and 
distinguished teachers—to assist schools and districts identified for 
improvement and corrective action as well as other districts and 
schools receiving Title | funds. 


Schools that do not make AYP for two consecutive years are identified for improvement 
and targeted for assistance and, if they continue to not make AYP, identified for increasingly 
rigorous interventions designed to bring about meaningful change in instruction and 
performance. In addition to providing more choices for parents of children attending identified 
schools, the law mandates corrective actions and, ultimately, the fundamental restructuring of 
any school that does not improve over an extended period. 


NCLB accountability requirements give parents of children in schools identified for 
improvement a new range of options. Parents with children in Title I schools that are identified 
for improvement, or corrective action or restructuring, may transfer their children to a non- 
identified school, including a public charter school, within their district. If they do so, the 
district must provide transportation to eligible students, using up to an amount equal to 20 
percent of the district’s Title I, Part A, allocation to pay for the transportation. If funds are not 
available to provide transportation to all eligible students, the district must give priority to low- 
income low-achieving students, though all students are eligible for transfer. Students from low- 
income families in Title I schools that are in their second year of improvement (or corrective 
action or restructuring) are also eligible to receive supplemental educational services. Students 
in these schools may choose either school transfers or supplemental services. These options are 
closely linked to the public reporting provisions that give parents information on which schools 
in their community are reaching their goals and which are not. The choice and supplemental 
services requirements of the law are intended to help enhance student achievement and provide 
an incentive for schools in need of improvement to improve. 


Overview of the Study 


Data collection for the study began in 2001-02, the last year of the NCLB’s predecessor, 
the Improving America’s Schools Act (ASA), and includes the first two years in which states, 
districts, and schools operate under NCLB. The study is designed to address how states, districts 
and schools are implementing accountability provisions under Title I, the alignment between 
Title I and state and district accountability systems, the assistance and incentives provided to 
Title I schools to help them improve, and student achievement in schools identified for 
improvement. 


Data collection for the evaluation consists of five components that span the three years of 
the study (additional information about the samples can be found in the appendix): 


¢ A survey of district Title I administrators in a nationally representative sample of 
approximately 1,300 districts that receive Title I funds. Districts were stratified 
according to size (enrollment), degree of poverty (based on the percentage of children 
living in poverty within each district), and geographic region. 


* See Exhibit 2 for definitions of size and poverty strata. 


¢ A survey of principals in a nationally representative sample of 739 Title I schools 
identified for improvement in 2001-02 drawn from the sampled districts. In 2002- 
03 the same schools were surveyed; however, since (1) the universe of identified 
schools changed and (2) this sample includes some schools that had moved out of 
improvement, the sample of identified schools in the analyses are no longer nationally 
representative of all identified Title I schools in the nation. However, surveying the 
same schools allows for tracking changes in these schools over time. 


* Case studies of 20 schools identified for improvement under Title I in 15 districts 
in five states. The five states were selected for variation on three critical accountability 
dimensions: state AYP definition, alignment of Title I and the general state 
accountability systems, and the state process for identifying schools in need of 
improvement. Within states, three districts were selected: a large urban district, one 
suburban district, and one rural district. Within each urban district, two elementary 
schools identified for improvement were selected. In the rural and suburban districts, 
one elementary school was chosen (often the only identified school in those districts). 
At each case study site, district and school employees and the parents of students in 
identified schools were interviewed. 


¢ State level interviews of state administrators and analyses of state accountability 
systems components. Key respondents include state Title I directors and 
accountability staff. 


The reader should be aware that throughout the report, school survey findings are generally 
based on the sample of Title I schools that continued to be identified for school improvement 
under Title I in 2002-03 (weighted N = 3,515, unweighted N = 374). The report refers to these 
schools as either “schools that continued to be identified” or “continuously identified schools.” 
In the few instances where findings apply to a larger set of schools, references to continuing 
identification are not made. Statistics are reported only when the unweighted sample was 20 or 
more. Moreover, all group differences that are reported as significant are statistically significant 
at p<.05. Details on the statistical tests and the standard errors of statistical estimates can be 
found in the appendix. 


Overview of the Report 


The findings for 2002-03 are organized around three topics related to the major 
accountability provisions of NCLB: (1) school improvement and the assistance provided to these 
schools, (2) public school choice and supplemental services, and (3) corrective actions and 
restructuring activities. Title I schools and districts become subject to these provisions when 
they are identified for improvement. But the implementation of these NCLB accountability 
requirements occurred within a broader accountability landscape that influences which schools 
are identified. As a backdrop to the findings presented in Chapters II-IV, the remainder of the 
Introduction describes the characteristics of schools and districts identified for improvement 
under Title I in 2002-03 as a basis for understanding the nature of the schools and districts that 
are the subject of this study. 


Title | Schools and Districts Identified for Improvement 


Under NCLB, schools that do not make AYP for two consecutive years must be identified 
for improvement. The consequences outlined in NCLB for identified schools and districts were 
to be applied during the law’s first full year, the 2002-03 school year, making identified schools 
and districts in 2002-03 the first to be subject to NLCB accountability requirements. Because 
states were developing AYP definitions following NCLB criteria during 2002-03, these 
definitions were not applied to identify schools and districts in that year. Instead, NCLB laid out 
transition rules for 2002-03: schools and districts were to be identified based on their 
improvement status as of Jan. 7, 2002—the day before NCLB was enacted. For example, a 
school in its first year of improvement as of Jan. 7, 2002, was to be identified as a school in its 
first year of improvement under NCLB’s accountability requirements. Similarly, a school in its 
second year of improvement as of Jan. 7, 2002, was to be identified as a school in its second year 
of improvement under NCLB. The exception to these transition rules was a school in 
improvement on Jan. 7 that had made one year of AYP in 2000-01 and could exit improvement 
status if it also made AYP based on its 2001-02 assessment results. States were also permitted to 
identify schools for improvement in 2002-03 that had missed AYP for the first time based on 
their 2000-01 assessment results (and thus were not identified for improvement as of Jan. 7, 
2002) and also missed AYP based on their 2001-02 assessment results. Similar transition rules 
applied to districts. In some cases, schools were not identified for 2002-03 until after the end of 
the 2002-03 school year and this study’s data collection. As a result of these factors, inferences 
about trends in numbers of schools and districts identified for improvement should be made with 
caution. 


Despite the transition rules, case study data at the school and district levels indicated that 
other issues contributed to the challenge of achieving widespread understanding of which 
schools were identified in 2002-03, such as changes in some state assessments, the presence in 
some states of parallel state accountability systems that use criteria different from AYP, and 
anticipated changes to state AYP definitions under NCLB. 


In 2002-03, 6,000 schools were identified for improvement under Title I, representing 
13 percent of all Title I schools nationwide. About 9,000 schools were identified in 2001-02.” 
The 6,000 identified Title I schools represent 13 percent of all Title I schools (see Exhibit 2). 
The decline in the number of Title I schools identified for improvement in 2002-03 was probably 
influenced by some of the factors noted above. 


Slightly less than half (46 percent) of the schools that had been identified under Title I 
as of 2001-02 continued to be identified in 2002-03. Fifty-one percent were no longer 
identified, 2 percent no longer received Title I funds, and a few (fewer than | percent) had closed 


” The first-year report for this study (ED 2004a) reported a smaller estimated number of identified schools for 
2001-02. The difference between the earlier estimate and the one above is due to slight differences in the sample 
and methods for estimation, with the estimate above drawing on a more complete data set than the earlier 
estimate. The differences between the two estimates for 2001-02 are not statistically significant (see appendix 
methods notes). 


or reorganized. Schools that exited improvement status did not differ from those that continued 
to be identified on any of the demographic variables examined: district size, school enrollment, 
district poverty level, school poverty level, or school level (elementary, middle, or high school). 


Among districts with identified schools, a higher percentage of identified schools were 
located in urban and high-poverty districts in 2002-03. Sixteen percent of districts had at 
least one identified Title I school (see Exhibit 2). Forty-seven percent of identified schools were 
located in urban districts, with the remainder split between suburban and rural districts (24 


Exhibit 2 


Title | Schools Identified for Improvement and All Title | Schools in 2002-03, 
by District Size, Location and Poverty Distributions 


Estimated 
number of Title | Percent of 
schools Percent of all schools 
identified for Title | identified Percent of all identified within 
improvement schools Title | schools categories 
District size, by student 
enrollment 
Small (200 to 3,503) 1,738 29 40 9 
Medium (3,504 to 10, 448) 883 15 24 8 
Large (10, 449 to 37,740) 1,246 21 17 15 
Very large (>37, 740) 2,135 36 19 24 
Total 6,002 100 100 13 
Location 
Rural 1,766 29 42 9 
Suburban 1,416 24 32 9 
Urban 2,820 47 25 23 
Total 6,002 100 100 13 
District poverty, by percent of 
children living in poverty 
Lowest poverty (<11 percent) 701 12 23 6 
Middle poverty (11 to 22 percent) 1,682 28 36 10 
Highest poverty (>22 percent) 3,618 60 41 18 
Total 6,002 100 100 13 


Exhibit reads: An estimated 1,738 Title I schools in small districts were identified for improvement in 2002-03, 
which represents 29 percent of all identified Title I schools in the nation. Small districts contained an estimated 
19,188 Title I schools overall, or 40 percent of all Title I schools. In small districts, an estimated 9 percent of Title I 
schools (i.e., 1,738 divided by 19,188) had been identified for improvement. 


Note: Totals may not add to 100 percent due to rounding. The denominator for computing the percentage of schools 
identified for improvement in each category (third column) is 6,002 schools (unweighted N = 3,366). The 
denominator for computing the percent of all Title I schools in each category (fourth column) is 47,490 schools 
(unweighted N = 17,487). The identification rate for schools was computed by dividing the number of identified 
Title I schools by the number of all Title I schools in each category. See the appendix for sample sizes and 
additional statistical information. 


Source: TASSIE Eligibility Dataset. Size and poverty designations are initial stratification variables; location was 
obtained from the 2000 CCD. 


percent and 29 percent, respectively). Identified schools were also concentrated in high-poverty 
districts (60 percent). However, 44 percent of districts with identified Title I schools were still 
located in small and medium districts with enrollments under 10,449 students (52 percent in 
2001-02). 


Fewer Title I districts had one or more schools identified for improvement under 
Title I in 2002-03 than in the previous school year. In 2002-03, 16 percent of Title I districts 
nationwide or an estimated 1,900 districts, had at least one identified school, as compared with 
26 percent or an estimated 2,900 districts in 2001-02.'° 


The majority of Title I districts with schools identified for improvement had only one 
school identified. Fifty-eight percent of districts with identified schools had only one Title I 
school identified for improvement in 2002-03 (see Exhibit 3). 


Exhibit 3 


Distribution of Identified Schools Across All Districts and Districts With 
Identified Title | Schools in 2002-03 


Number of identified Percent of districts with 
schools in districts Percent of all districts identified schools 
0 84 Not applicable 
1 10 58 
2 3 15 
3-4 2 13 
5-12 2 10 
13 or more <1 2 


Exhibit reads: Eighty-four percent of districts nationwide had no Title I schools identified for 
improvement in 2002-03; 16 percent therefore had identified Title I schools. Among all districts, 10 
percent nationwide had only one identified school; among districts with identified Title I schools, 58 
percent had just one identified school. 

Note: The percent of all districts is based on an N of 11,197 districts and the percent of districts with 
identified schools is based on an N of 1,824 districts. Percentages may not add to 100 percent due to 
rounding. 


Source: TASSIE district survey. 


Among the districts that had at least one Title I school identified for improvement in 2002- 
03, 39 percent had less than a quarter of their schools identified, 26 percent had 25 to 50 percent 
of their schools identified, 23 percent had 50 to 75 percent of their schools identified, and 12 
percent had over three-quarters of their schools identified. When measuring the concentration of 
identified schools within districts, it should be noted that districts with small numbers of schools 


'° The first-year report for this study (ED 2004a) reported a smaller estimated percentage of identified schools for 
2001-02. The difference between the earlier estimate and the one above is due to slight differences in the sample 
and methods for estimation, with the estimate above drawing on a more complete data set than the earlier 
estimate. The difference in the two estimates for 2001-02 are not statistically significant (see appendix methods 
notes). 


are more likely to have higher concentrations of identified schools if they have any schools at all 
identified for improvement. That is, a small district that has one of its two schools identified for 
improvement is in the same category as a large district in which 50 of its 100 schools are 
identified (i.e., both have 50 percent of their schools identified). 


Districts with identified Title I schools in 2002-03 were more likely to be poor and 
urban than districts without identified schools but no different in their total enrollments. 
Districts with identified schools were more likely than districts without identified Title I schools 
to have a high percentage of students from low-income families (see Exhibit 4). (Forty-two 


Exhibit 4 


Demographic Characteristics of Districts With and Without 
Identified Title | Schools in 2002-03 


Districts with identified Districts without 
Title | schools identified Title | schools 
Percent of Percent of 
Number total Number total 
District size, by student 
enrollment 
Small (200 to 3,503) 1,035 57 7,503 80 
Medium (3,504 to 10,448) 434 24 1,507 16 
Large (10,449 to 37,740) 283 16 342 4 
Very large (>37,740) 72 4 54 1 
Location 
Urban 268 15 276 3 
Suburban 590 32 2,457 26 
Rural 967 53 6,672 71 
District poverty, by percent of 
children living in poverty 
Lowest poverty (<11 percent) 345 19 3,627 39 
Middle poverty (11 to 22 percent) 718 39 3,643 39 
Highest poverty (>22 percent) 761 42 2,135 23 
Total 1,824 100 9,405 100 


Exhibit reads: The estimated number of districts nationwide with one or more identified Title I schools that 
were small districts was 1,035 which represented 57 percent of all Title I districts that had an identified 
Title I school in 2002-03. An estimated 7,503 districts nationwide without any identified Title I schools 
were small districts which represented 80 percent of all districts that did not have an identified Title I 
school. 


Note: Totals may not add to 100 percent due to rounding. 


Source: TASSIE Eligibility Dataset. District size and poverty designations are initial stratification 
variables; location was obtained from the 2000 CCD. 
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percent of districts with identified Title I schools compared with 23 percent of districts without 
identified Title I schools had a high percentage of students, greater than 22 percent, from low- 
income families.) Districts with identified Title I schools were more likely than districts without 
identified schools to be located in urban areas (15 percent vs. 3 percent) and less likely to be 
located in rural areas (53 percent vs. 71 percent). Districts with identified Title I schools did not 
differ from other districts in their total enrollment. At the same time, a high percentage of 
districts with identified Title I schools are small (59 percent) and located in rural areas (52 
percent). 


Across the nation, few districts were identified for improvement in 2002-03. State- 
level interviews indicated that 33 states had a process for identifying districts in place, seven 
states had not yet established such processes, three state respondents were unsure of the status of 
Title I district identification, and data were unavailable in seven states regarding district 
identification.'’ Of the states with identification systems in place, only eight had identified 
districts for improvement under Title I in 2002-03. 


¢ About two-thirds (64 percent) of districts reported that their state had begun to review 
the progress of districts towards achieving AYP (similar to the percentage in 2001-02); 
19 percent of districts were not sure whether or not the state was doing so. 


¢ Of districts in states reviewing district AYP, the majority (84 percent) reported that 
their district had made AYP in 2002-03; 9 percent reported not making AYP; and 7 
percent did not know whether or not they had done so. 


The number of districts identified for improvement decreased from 2001-02 to 
2002-03. Among the 64 percent of districts that reported that their state had begun to identify 
districts for improvement, only 5 percent (an estimated 373 districts nationwide) reported that 
they had been identified for improvement under Title I, compared with 15 percent in 2001-02. 
Identification for improvement did not vary by district poverty level, but there were significant 
differences by district size. While the total percentage of districts identified for improvement 
was 5 percent, among very large districts 11 percent were identified, among large districts 5 
percent were identified, among medium districts 3 percent were identified, and among small 
districts 6 percent were identified. This is the reverse of what happened in 2001-02, when 
significant differences were noted in regard to district poverty level but not district size. 


For accountability systems to be effective, local educators must understand the 
accountability criteria and must be aware of whether or not their schools have been 
identified for improvement, and in 2002-03 educators’ and the public’s understanding of 
what it meant to be identified for improvement was not always clear. A central principle 
underlying the Title I accountability system is that local educators have a clear understanding of 
the process used to hold schools accountable. Without such an understanding, accountability 
systems cannot be expected to have an impact. 


'' Hawaii is a unique case in that the SEA is also the district, and no system was in place for the SEA to identify 
itself. (The same is true of the District of Columbia, although it is not a state.) 


1] 


In 2002-03, just over three-fourths (76 percent) of district administrators reported that they 
understood all or most of the elements of the AYP definition for Title I schools. However, 
principals’ reports on their schools’ improvement status were not always consistent with the 
state’s or district’s categorization of the school. 


¢ In 30 percent of the continuously identified Title I schools, principals were not aware 
that their schools were identified for improvement in 2002-03. In 2001-02, 41 percent 
of principals in identified Title I schools reported that they were not identified for 
improvement or did not know if they had been identified for improvement. 


¢ In 19 percent of schools that were no longer identified, principals reported that the 
school was still identified for improvement in 2002-03. (This was not an issue in 2001- 
02 since all of the schools responding to the survey were identified for improvement.) 


Case study data provided some examples of the confusion caused by pending transitions to 
new AYP definitions under NCLB and assessment systems as they existed in 2002-03. For 
example, Maryland allowed districts to apply local criteria to assess schools’ AYP as the state 
transitioned to a new AYP formula so the definitions were not consistent across districts. In 
addition, the state’s transition from one assessment to another also caused some confusion about 
AYP measures. In two large districts in Maryland, most school staff in identified schools 
generally knew that their school had been identified, but they were less certain about AYP 
elements and how the school could exit improvement status. When asked about the school’s 
performance target, one principal stated, “Who knows what it is. ... We don’t know what 
measure we are going to be judged on...at some level it doesn’t really matter, we know what we 
need to do.” 
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Il School Improvement and District Assistance in Identified Schools 


To achieve the goal that all students reach proficiency by 2013-14, NCLB calls for greater 
accountability, coupled with increased assistance to schools in which too many children are not 
meeting state standards for proficiency in reading and mathematics. NCLB lays out specific 
steps that states and districts must take to assist schools in need of improvement. States must 
establish statewide systems of support that provide assistance directly to schools. Districts must 
provide ongoing technical assistance as schools develop and implement their school plans. In 
particular, districts must help schools analyze student achievement data, develop plans for 
improvement, revise their budgets so that resources are effectively allocated to the activities 
most likely to increase student academic achievement, implement professional development, and 
put in place instructional practices that have shown evidence of effectiveness. Schools, in turn, 
are expected to develop and implement two-year plans that provide road maps for their efforts to 
improve curriculum and instruction and raise student achievement. 


In 2002-03, schools in the study sample that continued to be identified for improvement 
under Title I (“continuously identified schools”) were engaged in multiple improvement 
activities, and many districts and states provided technical assistance to help these schools meet 
their goals. However, during the 2002-03 school year not all states and districts had school 
support systems in place, as called for under NCLB, and only a fraction of identified schools 
reported receiving such assistance. 


As in 2001-02, both identified schools and their districts emphasized school planning, 
looked at achievement data, and matched curriculum to standards and assessments in 2002-03. 
These activities are consistent with those promoted by NCLB and they lay the groundwork for 
changes inside classrooms, but they do not substitute for such changes. Many schools had begun 
new programs, especially in reading, but far fewer had received professional development and 
support for their efforts. Improvement efforts to this point were due primarily to continuing state 
and district efforts that (1) began before 2002-03, the year that NCLB went into effect, and (2) 
applied to all schools or all low-performing schools, not just those formally identified for 
improvement. The majority of districts were providing some form of assistance to identified 
schools, but many, especially small districts, did not provide any additional assistance or 
dedicate extra staff to support school improvement. 


This section elaborates on these findings regarding systems of support, looking first at 
which entities provided and received support and then at the focus of assistance and 
improvement under which findings on planning and data use, new curricula, and professional 
development are described. The last two subsections report how districts targeted assistance and 
support and the variations in assistance efforts by district characteristics. 


State and District Systems of School Support 


NCLB assigns responsibility to both states and districts for supporting improvement 
activities in identified schools. The law requires that states develop systems of school support 
that provide assistance to schools identified for improvement and schools in corrective action 
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and restructuring status. These support systems must include school support teams that work 
with identified schools to design, implement, and monitor school improvement plans, as well as 
with distinguished teachers and principals recruited from schools that have been especially 
successful in improving student achievement. Because districts also have special responsibility 
for providing assistance to identified schools under NCLB, they are another important source of 
support and assistance. 


Most states provided some type of assistance to their identified schools in 2002-03; 
however, more than a third of the states had not organized support teams or distinguished 
educators to provide assistance to identified schools as required under NCLB. Twenty- 
three of the 50 states plus the District of Columbia reported that they were operating school 
support teams, as required under NCLB (see Exhibit 5). Typically, these school support teams 
assisted schools by conducting needs assessments, developing school improvement plans, and 
monitoring the implementation of those plans. Nineteen of 51 states reported that they had 
assigned distinguished educators to schools. Like school support teams, distinguished educators 
typically assisted with the development of school plans but sometimes worked closely with 
teachers on improving instruction. Twenty-eight states provided assistance to identified schools 
either through school support teams, or through distinguished teachers, or through both. In more 
than a third of states (19), however, these two types of assistance were not available to any 
schools. 


Exhibit 5 
State Reports of Assistance Provided to Identified Schools 
Number of states reporting 
Assistance Assistance not Data notavailable 
provided provided 

School support team 23 24 

Distinguished educator 19 26 

Other types of 

assistance oa e ° 


Exhibit reads: Twenty-three states reported that they assigned school support teams to at least some 
identified schools. Twenty-four states reported that they did not provide this type of assistance. In four 
states, these data were not available because the respondent did not know or the state did not respond to 
requests for an interview. Sample includes 50 states and the District of Columbia. 


Source: TASSIE state survey. 


Thirty-nine states reported that they also provided other types of assistance to their 
identified schools (see Exhibit 5). These other types of assistance most often included technical 
assistance to identified schools, professional development, and assistance from regional 
agencies. Typically this assistance was offered via phone consultation or in regional training 
events, rather than in-person and on-site, as was the case with school support teams and 
distinguished educators. For example, Arizona’s state department of education hosted a three- 
day workshop on using test score data to develop school improvement plans for school 
improvement teams from across the state. Maryland offered two training sessions on closing the 
achievement gap for minority students to teams from identified schools. 
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NCLB requires that, at a minimum, school support teams, distinguished educators and 
other types of assistance be provided to identified schools in each state. Though most states (46) 
had in place at least one of the required elements of a statewide system of support in 2002-03, 
very few had all of the required elements in place. Twelve states reported that they had provided 
assistance to identified schools via all three elements: school support teams, distinguished 
educators, and other forms of assistance. 


The fraction of identified schools receiving assistance from school support teams and 
distinguished educators in 2002-03 varied across states, as did the intensity of support 
provided which in part was based on the level of school need. Only 16 of the 23 states that 
had school support teams reported on the number of schools served. In seven states, either the 
state did not keep records of the number of schools served or the respondent did not have access 
to this information. Of those states that did report on the number of schools served, 11 indicated 
that school support teams reached all or nearly all of the identified schools in the state (between 
75 and 100 percent). Five states reported, however, that their school support teams reached 
fewer than half of identified schools. Distinguished educators appear to have reached a smaller 
fraction of schools in many states. Specifically, 6 of 11 states reported that distinguished 
educators reached fewer than half of identified schools. Five states reported that distinguished 
educators reached more than half of identified schools. In 6 of the 19 states that had 
distinguished educators, information about the number of schools served was not available. 


In some states, school support teams and distinguished educators were assigned to 
identified schools to provide intensive support and assistance in developing, implementing, and 
monitoring schools’ improvement plans; in other states, support from these sources was more 
limited. In Louisiana, for example, the state worked with districts to select and train District 
Assistance Teams (DATs), which visited schools throughout the year to provide assistance with 
planning, monitor the implementation of the plan, and provide quarterly status reports to the 
district and state. Louisiana supplemented DATs with curriculum specialists who spent up to a 
semester in identified schools, and distinguished educators who spent a full year in corrective 
action schools. Washington’s Focused Assistance Program supported a facilitator in each 
participating school who worked one-third time with the school to develop and implement an 
improvement plan (see Exhibit 6). In other states, for example, Arizona, external evaluators 
helped with the development of school plans, but made fewer trips and spent much less time in 
schools. 


In 2002-03, grant programs funded by Title I school improvement allocations and other state 
and federal funding streams were another key element in state systems of school support. The 
majority of districts with identified schools (57 percent) reported that their state provided 
identified schools with grants for school improvement. For example, Michigan allocated Title I 
school improvement funds to its lowest-performing Title I schools, known within the state as 
“intensive” schools. Schools applied for grants, which were limited to spending on professional 
development. Identified schools in one large urban district in Michigan received $75,000-95,000 
in funding for new professional development. Other states encouraged identified schools to apply 
for Comprehensive School Reform (CSR) and Reading First funds to secure additional resources 
for improvement activities. 
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Exhibit 6 
Washington’s Focused Assistance Program 


In Washington during the 2002-03 school year, the Focused Assistance program provided intensive 
support for school improvement planning and implementation to schools identified for improvement 
under Title |. First, an educational audit team visited participating Title | identified schools for three to 
four days to conduct a needs assessment. The audit teams were made up of five to six people 
including: a team leader (typically a former administrator selected for writing and communication 
skills), a superintendent, a staff member from the local education service district, an experienced audit 
team member, and a district or state staff person who has experience with the student population at the 
school. This visit concluded with the presentation of a written assessment of the schools strengths and 
weaknesses. Second, the state provided a facilitator to work with the school to develop a school 
improvement plan based on the findings of the audit team. Facilitators were paid for 78 days during the 
year, most of which were spent at the school site; schools also received money for professional 
development through the program. Participation in the program was voluntary and was limited to 
schools that had been identified for improvement under Title |. As of 2002-03, the program had 
reached 38 of the 50 identified schools in the state, in 31 districts. 


Although some states had established mechanisms for providing support to identified 
schools, many identified schools turned first to districts for assistance in mounting meaningful 
improvement efforts. District assistance to Title I identified schools included placing additional 
staff in schools to support teacher development; and various kinds of consultants, similar to the 
school support teams and distinguished educators operated by states, who provided consultation 
and other support for data analysis, planning, and budgeting. 


Assigning additional school-level staff to identified schools was a key district strategy for 
supporting school improvement reported in 2002-03. Most districts with identified schools 
reported that they provided those schools with some form of additional staffing, in addition to the 
standard complement of classroom teachers. For example, nearly three-quarters of districts with 
identified schools reported that they assigned additional staff to support instructional 
improvement, and the same number of districts reported that they assigned additional staff to 
provide additional instruction to students. One third of districts with identified schools reported 
that they assigned those schools additional full-time staff whose primary responsibility was to 
support teacher professional development. The majority of districts (70 percent) also reported 
that they provide Title I identified schools with additional professional development or special 
access to professional development resources. 


Among continuously identified schools, 21 percent reported that they had been assigned 
full-time school-level staff to support teacher development. Just over half (58 percent) reported 
that they had received additional professional development or special access to professional 
development resources. A description of district-sponsored professional development and of the 
role of these special school-based staff follows in the section on professional development below. 


Most, but not all, districts provided on-site assistance to their identified schools in the 
form of school support teams, distinguished teachers, or principal mentors. Like some 
states, some districts had assigned teams of administrators and teachers to assist identified 
schools with the development of school improvement plans, use of data, and other activities 
related to instructional improvement. Slightly more than half of districts (52 percent) reported 
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that they had fielded school support teams for identified schools. Nearly a third (29 percent) 
reported that they had assigned a principal coach or mentor to identified schools; 15 percent of 
districts had assigned distinguished teachers (see Exhibit 7). Overall, nearly two-thirds of 
districts reported that they provided identified schools with at least one of these forms of on-site 
assistance. In addition, districts reported that they provided school support teams, distinguished 
teachers, and principal mentors to identified schools much more often than states did. 


Exhibit 7 


District Reports of Assistance Provided to Identified Schools, 
Among Districts With Identified Schools 


Percent of districts reporting assistance provided 


Assistance provided by district | Assistance provided by state 


School support team 52 32 
Principal mentor or coach 29 10 
Distinguished teacher 15 8 


Any of these forms of 
assistance (i.e., school 
support team, principal 61 39 
mentor, or distinguished 
teacher) 


Exhibit reads: Among districts with identified schools, 52 percent of districts reported that they provided 
school support teams to identified schools; 32 percent reported that the state provided school support team to 
identified schools in the district. 


Note: See the appendix for sample sizes and additional statistical information. 


Source: TASSIE district survey. 


District-sponsored on-site assistance could include a wide range of activities. For example, 
one large suburban district assigned an instructional specialist to visit each Title I school once a 
week to monitor school improvement plans, conduct parent outreach, and monitor the use of 
Title I funds at the school. In Louisiana, the DATs described above visited schools at least 
quarterly to help with gathering and interpreting of data to create a school improvement plan, 
using tools developed for this purpose by the state. 


Other case study districts assigned district staff to work with identified schools, in addition 
to their regular responsibilities. For example, one rural district assigned each of its curriculum 
specialists to serve on the school improvement team of one school in the county. Other districts 
created new lines of reporting and supervision for principals of low-performing and identified 
schools. Principals and school improvement team members from these schools met once or 
twice a month with district leadership teams, assistant superintendents, or chief academic 
officers, to review school improvement plans and progress on implementation of those plans. 
Reports from case study schools suggest wide variations in how much assistance schools 
received as well as whether the assistance was perceived as helpful by school staff. 
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Many continuously identified schools had no access to state or district on-site 
assistance, either because they were in districts and states that did not provide it or because 
states or districts did not serve all of their identified schools. Among schools that continued 
to be identified for improvement in 2002-03, one-quarter reported that they had received help 
from a school support team within the last year and fewer still reported help from a principal 
mentor or coach or from a distinguished teacher (see Exhibit 8). Altogether, slightly more than a 
third of continuously identified schools reported that they received one or more of these forms of 
assistance, whereas nearly two-thirds reported that they had received none. 


Exhibit 8 


School Reports of On-site Assistance, 
Among Schools That Continued to Be Identified for Improvement 


Percent of schools reporting that they received assistance 


Schools in districts Schools in districts 
All schools where the district where the state 
provides assistance provides assistance 


School support team 25 35 32 
Principal mentor or coach 20 39 4 
Distinguished teacher 13 2 


Any of these forms of 
assistance (i.e., school 
support team, principal 38 48 37 
mentor, or distinguished 
teacher) 


Exhibit reads: Twenty-five percent of all schools that continued to be identified for improvement in 2002-03 
reported that they received help from a school support team. Where the district provided school support teams, 35 
percent of schools reported that they had received help from one. Where the state provided school support teams, 
32 percent of schools reported receiving help from one. 


Notes: ** indicates there are too few cases in the cell to generate a reliable estimate. See the appendix for sample 
sizes and additional statistical information. 


Source: TASSIE school survey. 


Even when continuously identified schools were located in districts or states that provided 
some form of on-site assistance, the proportion reporting that they received that assistance was 
still low. Among schools in districts that fielded school support teams, only a third reported that 
they received assistance from a team (see Exhibit 8). Similarly, only a third of the continuously 
identified schools located in states that fielded support teams reported that they received 
assistance from a team. Although a majority of districts and states offered school support teams, 
distinguished teachers, and principal mentors to at least some of their identified schools, the 
evidence suggests that these forms of on-site assistance were, in many cases, not available to 
every identified school. In several case study sites, districts or states had targeted on-site 
assistance to a subset of the lowest-performing identified schools, or to a group of schools that 
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had volunteered to participate in a school improvement initiative. In states or districts like these, 
some identified schools did not receive the assistance that was provided to others. 


A small number of identified schools had no access to school support teams, distinguished 
teachers, or principal mentors because they were located in jurisdictions where neither the 
district nor the state provided these types of support for school improvement. For example, 17 
percent of identified schools had no access to school support teams because neither their district 
nor their state provided them. Thirty-four percent of principals had no access to mentors 
because neither the district nor the state provided them. Nine percent of identified schools were 
located in districts where neither the state nor the district offered any of these forms of 
assistance. 


Assistance from school support teams was provided to fewer continuously identified 
schools in 2002-03 than the previous year. In 2002-03, slightly more than a quarter (28 
percent) of the principals in continuously identified schools in the longitudinal sample reported 
that they had received help from a school support team, down from 46 percent in 2001-02. This 
drop is surprising, because these schools had been identified for one additional year, and 
therefore should have been in even greater need of assistance. The percentage of principals 
reporting that they had received help from a mentor or coach remained unchanged, but low, as 
did the percentage reporting that they had received help from a distinguished teacher. In several 
case study sites, increasing budget deficits and pressure to trim expenditures by cutting central 
office staff had forced districts to cut both the number of staff assigned to serve identified 
schools and the number of schools served. 


Focus of District Assistance and School Improvement Efforts 


School improvement efforts and district assistance focused overwhelmingly on 
increasing the use of achievement data and matching curriculum and instruction with 
standards and assessment. Consistent with NCLB’s expectation that schools and districts use 
data to identify students who are not on track to reach standards and to assess the effectiveness 
of existing instructional programs, 77 percent of districts with at least one identified school 
reported a major focus on increasing the use of student achievement data to inform instruction 
and school improvement (see Exhibit 9). NCLB also holds districts and schools accountable for 
student achievement on assessments that are aligned with academic content standards. Eighty- 
six percent of districts with identified schools placed a major focus on aligning curriculum and 
instruction with standards and assessments. 
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Exhibit 9 
Percent of Districts Placing a Major Focus on Strategies for Improving Low-Performing 
Schools, Among Districts With Identified Schools 
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Exhibit reads: Among districts with identified schools, 86 percent reported that matching curriculum and 
instruction with standards, assessments, or both, was a major focus of district resources for low-performing 
schools. 


Note: See the appendix for sample sizes and additional statistical information. 


Source: TASSIE district survey. 


Continuously identified schools generally relied on the same improvement strategies 
in both 2001-02 and 2002-03. The vast majority of schools had a written school improvement 
plan (93 percent), used data to inform their improvement efforts (85 percent), took steps to align 
curriculum to standards and assessments (80 percent), and reported some teacher collaboration 
(75 percent).'? Smaller percentages of schools reported adoption of new curricula and reform 
models because a given school was more likely to adopt only one of these (see Exhibit 10). 
Compared with 2001-02, more schools used data to inform their improvement efforts in 2002-03 
(85 percent compared with 75 percent); otherwise, use of strategies did not change. 


The following subsections elaborate the above findings on: school planning and uses of 
data, curriculum alignment, new curricula and school reform models, and professional 
development. 


"? The definition of “teacher collaboration” as a school improvement strategy is described in the appendix. 
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Exhibit 10 


School Improvement Strategies, 
Among Schools That Continued to Be Identified for Improvement 


Percent of districts 
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Exhibit Reads: Among schools that continued to be identified for improvement in 2002-03, 93 percent reported 
having a written school plan. 


Note: See the appendix for sample sizes and additional statistical information. 


Source: TASSIE school survey. 


School Planning and Use of Data 


NCLB requires the development or revision of school improvement plans designed to 
address the problems that caused the school to be identified. In addition to requiring school 
plans, NCLB calls for districts to assist schools in: (1) analyzing student achievement data; (2) 
identifying research-based professional development and instructional strategies; and (3) 
allocating their resources more effectively. 


Almost all continuously identified schools developed school improvement plans and 
received district assistance consistent with NCLB requirements; however, only about half 
of those schools monitored progress toward their plans’ goals in 2002-03. Consistent with 
NCLB requirements, nearly all districts with identified Title I schools reported supporting 
schools in the development of school plans. Districts reported that they assigned staff to provide 
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Title I schools identified for improvement with three specific types of assistance called for under 
the law: help with analyzing data (94 percent); help with identifying research-based 
improvement strategies (89 percent); and help with allocating resources more effectively (72 
percent). In addition, most districts had provided other types of support for school planning and 
data use, including help identifying research-based strategies, additional data analysis, and help 
with monitoring progress toward goals. 


In many cases, the range of assistance provided by districts with regard to planning and 
data use was extensive. Nearly half (44 percent) of districts with identified schools reported that 
they provided schools with technical assistance in all 10 of the areas related to data use and 
planning included in the TASSIE district survey.'* Only 13 percent of districts reported that 
they provided assistance on 5 of those 10 items or fewer. 


As required under NCLB, most continuously identified schools (93 percent) had written 
school improvement plans for the 2002-03 school year. Many schools made frequent reference 
to their improvement plan and used data to monitor progress toward goals and to revise the plan 
on a regular schedule, while others did not. For example, slightly more than half of principals 
(56 percent) reported that educators at their school were monitoring progress toward goals 
established in their school plans at least quarterly in 2002-03—a reduction from 75 percent in 
2001-02. About a quarter (23 percent) reported monitoring progress two or three times a year 
and another 23 percent reported monitoring their progress annually or once every two years. 


Almost all (96 percent) continuously identified schools reported using their improvement 
plans to design or choose professional development, 83 percent reported using their plan to 
select and implement new curricula and instructional programs, and 51 percent reported using 
their plan to select and implement a school reform model. However, case study schools suggest 
these figures likely reflected post-hoc rationales for choices about professional development and 
curriculum. In these schools, both school plans and decisions about professional development 
and curriculum were influenced by the same forces, such as district mandates and eligibility 
requirements for grants. 


The extent to which the development and use of the school plan were collective activities 
also varied by school. In some of the case study schools, all educators were aware of the school 
plan and the actions being taken to accomplish its goals. For example, the schools in one urban 
district had school improvement teams and committees that met monthly, looked at data, and 
revised plans accordingly. In others, only a few teachers were aware of the contents of the plan. 


A majority of continuously identified schools had access to disaggregated 
achievement data, but most principals did not identify disaggregated data as among the 
most influential sources of data for school planning. NCLB requires schools to be held 
accountable for the performance of subpopulations of students; as a result, access to 
disaggregated student achievement data is needed. Roughly two-thirds of principals of identified 
schools reported access to data disaggregated by special populations such as limited English 


'? Survey items about district support for planning and data use were based on two questions that asked about the 
types of technical assistance provided to schools and whether professional development addressed topics about 
data use. The 10 items in these two questions are listed in the appendix. 
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proficient and special education students, or disaggregated by demographic characteristics such 
as poverty level and race or ethnicity. However, only about a quarter of these principals reported 
that these data had served as among the most influential sources of data for school planning in 
2002-03. 


Schools that continued to be identified for improvement in 2002-03 were increasingly 
focused on the use of student achievement data to inform instruction and school 
improvement. The absence of attention to written school plans did not signify lack of attention 
to data. The percent of schools reporting that they placed a major focus on the use of student 
achievement data rose from 75 percent in 2001-02 to 86 percent in 2002-03. This increase was 
consistent with districts’ continuing focus on the use of data (see Exhibit 9). As the case study 
schools illustrated, schools were using data in multiple ways, from schoolwide and grade-level 
planning, to diagnosing the needs of individual students; however, diagnostic uses received less 
attention. 


The case study schools suggest that the use of diagnostic data for instructional decisions 
occurred but required a set of conditions that were not widespread: the availability of useful 
diagnostic assessments, district leadership and support, and the knowledge of how to tailor 
instruction on the basis of the results of the data (see Exhibit 11 for a case of exemplary data 
use). NCLB calls on districts to administer assessments that are useful in assessing the progress 
of low-achieving students and in diagnosing their learning needs. Most districts and a majority 
of schools reported availability of assessments other than annual tests: 


¢ Principals at more than half (56 percent) of the schools that continued to be identified 
reported that they had been provided with classroom-embedded assessments that 
aligned with state standards. 


¢ Most districts with identified schools (88 percent) reported administering local 
assessments. 


¢ Of these districts, 89 percent reported doing so in order to identify students who needed 
tutoring or other special interventions, and 92 percent reported doing so in order to 
inform instructional decisions. 


Exhibit 11 
School Uses of Data 


One case study school in a suburban district with overall low poverty exemplified extensive use of data. 
In fact, the school’s primary improvement strategy rested on close attention to many types of 
assessment data: classroom assessments, quarterly district assessments, the state assessment, and a 
nationally normed standardized test. Teachers identified students performing below grade level and 
matched those students with interventions (usually additional instruction) designed to address their 
particular weaknesses. Grade level teams met quarterly with resource teachers and the reading and 
math support teachers to go over data for each child. With only a small percentage of low-performing 
students, they were able to discuss each student who was not making progress and determine whether 
a new intervention strategy was needed. Behind this activity was a district requirement that schools 
develop individual student support plans for every student below grade level. The recordkeeping was 
helped by the district's decision to give networked laptop computers to teachers who used them to 
keep progress reports and student support plans, as well as grade books. 
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Curriculum Alignment 


Aligning curriculum with academic content standards and assessments was an 
increasingly common focus of district assistance and school improvement. In addition to 
focusing on the use of student achievement data to inform improvement efforts, most districts 
also reported that matching curriculum and instruction with standards, assessments, or both, was 
a major focus of district resources in 2002-03 (see Exhibit 9). In fact, among districts 
responding to the survey in both years, the percent characterizing curriculum alignment as a 
major focus area rose in 2002-03 (to 86 percent, from 73 percent in 2001-02). Similarly, 80 
percent of principals of continuously identified schools reported a major focus on curriculum 
alignment in 2002-03, similar to reports from the previous year. 


Consistent with this focus, almost all districts (91 percent) reported that they had sponsored 
professional development for identified schools on the topic of ensuring alignment between 
curriculum and instruction and standards (see Exhibit 15). Similarly, 79 percent of principals of 
continuously identified schools reported providing professional development aimed at ensuring 
alignment, and 55 percent of these principals identified curriculum alignment as one of three key 
areas for professional development. 


Districts and schools also pointed to a range of support provided by districts on curriculum 
alignment, in addition to professional development (see below): 


Sixty-nine percent of continuously identified schools reported that their state or district 
had provided curriculum guides with standards, frameworks, and pacing sequences. 


Fifty-five percent of continuously identified schools reported that they had been 
provided with documents mapping out the alignment of required textbooks and 
instructional programs with standards. 


A majority of districts provided each of the following kinds of support for alignment of 
curriculum and instruction with state or district standards: professional development 
on aligning curriculum with state or district standards and assessments, local content 
standards, curriculum guides, model lesson plans, aligning textbooks and instructional 
materials with standards and assessment, and regular checks of standards 
implementation in classrooms. Twenty-nine percent of districts reported that they 
provided all or nearly all of the types of support for alignment included on the TASSIE 
district survey (at least 9 out of 10 items).'* Nearly three-quarters of districts (73 
percent) provided at least a moderate range of support for curriculum alignment (at 
least 6 out of 10 items). 


In the case study schools, teachers cited benefits from curriculum alignment, but many 
reported inadequate time and support for translating alignment efforts into instructional change 
(see Exhibit 12). In addition, educators struggled with the need to align their curriculum with 


" Survey items about district support for curriculum alignment were based on two questions that asked about the 
types of technical assistance provided to schools and whether professional development addressed topics about 
curriculum alignment. The 10 items in these two questions are listed in the appendix. 
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multiple sets of standards and assessments. Some schools operated under both district and state 
standards, and their students took a combination of state and nationally normed tests as well. 


Exhibit 12 
Challenges of Curriculum Alignment 


Efforts to align curriculum and instruction with standards and assessments pose challenges to districts 
and schools. In Washington, for example, schools aligned to the state standards, the state’s criterion- 
referenced assessment, and a norm-referenced test. One district created documents available on CD- 
ROMs using commercial software that were described as linking every competency and objective at 
every grade level to one or more of these standards and assessments. However, few teachers used 
these materials—in large part because they were implementing highly structured reading programs that 
left little room to make adjustments. Another reason alignment documents and activities were limited in 
their effect on instruction was that they were not accompanied by adequate support. In another district, 
a teacher noted the difference between general discussions of alignment at the district level versus 
knowing how to meet the objectives: “Last week a district person was out talking about standards and 
how to meet them with different parts of the curriculum, but it’s more helpful to have that conversation 
in a grade level meeting. We need more supports to facilitate conversations about how to reach our 
objectives.” 


New Curricula and School Reform Models 


NCLB charges districts with assisting schools in identifying and implementing 
instructional strategies that will strengthen the core academic subjects in the school and that have 
proven effective in addressing the issues that caused the school to be identified. School plans 
and curriculum alignment represent first steps on the path to instructional improvement in the 
classroom. Adopting new instructional programs and providing related professional 
development are additional strategies that are often used to make changes that are one step closer 
to the classroom and student learning. 


In 2002-03, many schools were adopting new curricula, using school reform models or 
adding a variety of new supplemental instructional programs. Almost three-fifths of 
principals (58 percent) reported they had adopted a new language arts curriculum in the previous 
three years, and close to half (48 percent) of continuously identified schools had adopted a new 
mathematics curriculum in the previous three years. In many cases, these adoptions had been in 
response to district requirements that increasingly required all schools to adopt specific language 
arts or mathematics programs. In fact, 53 percent of districts reported requiring identified 
schools to adopt a new language arts curriculum and 40 percent required adoption of a new 
mathematics curriculum. 


The case study sites and school survey data suggest a trend among districts in centralizing 
decisions about curriculum and professional development to ensure alignment with state 
standards and assessments—decisions that previously had been delegated to individual schools 
during a decade characterized by school-site management. For example, one urban district 
leader characterized the district as recentralizing in order to provide low-performing schools 
with “higher yield strategies” (e.g., research-based instructional strategies) and to ensure 
common curriculum and assessments across schools. Of those schools implementing a new 
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language arts or mathematics curriculum, half (51 percent and 49 percent, respectively) reported 
that their district required them to do so. 


Many schools that continued to be identified (45 percent) were also using school reform 
models. Use of a school reform model was rarely (19 percent) prompted by a district 
requirement. 


Principals of identified schools cited reasons of instructional consistency for adopting 
new curricula, not Title I identification. Taking NCLB’s call for aligned state standards, 
assessments, and accountability systems one step further, schools were adopting new curricula as 
a means of aligning their curricula with academic content standards and assessments and of 
standardizing curriculum across classrooms. As the reason for new curriculum adoptions, 
roughly three-fourths of principals in identified schools that adopted new language arts or 
mathematics curricula cited the need to support instruction consistent with state or district 
standards and assessments and to establish consistency across classrooms and/or grade levels. In 
contrast, fewer than one-fourth reported that Title I identification played a role in adopting new 
language arts or mathematics curricula. 


Of the continuously identified schools that adopted new reading or mathematics 
curricula or school reform models, most reported emphasizing related professional 
development or receiving a minimum level of related district assistance; fewer reported 
receiving both.’> For example, of the almost 6 in 10 schools (58 percent) that continued to be 
identified and reported adopting a new language arts program in the last three years (1.e., 
between 2000-01 and 2002-03), 85 percent reported either emphasizing related professional 
development or receiving a minimum level of related district assistance (see Exhibit 13). At the 
same time, less than half (43 percent) reported receiving both. Similarly, almost three-quarters 
(71 percent) of schools adopting a new mathematics curriculum (48 percent) reported either 
emphasizing related professional development or receiving a minimum level of related district 
assistance while 11 percent reported receiving both. 


Although fewer than half of the continuously identified schools reported implementing 
new mathematics curricula and only two in five of those reported emphasizing related 
professional development, the frequency with which schools reported emphasizing professional 
development in mathematics increased significantly in 2002-03. The percentage of principals 
who reported placing emphasis on professional development in mathematics started low in 2001- 
02 (35 percent of those who reported that educators participated in mathematics professional 
development) but increased to 52 percent in 2002-03. In contrast, in both 2001-02 and 2002-03, 
the percentage of principals who reported emphasizing professional development in language 
arts was high (78 and 80 percent, respectively). 


The gap between the percent of schools implementing new curricula and those receiving 
related professional development, assistance, or both, suggests that schools may not have had 
access to all the help they needed to put new instructional programs in place. Adopting a new 
curriculum can be an important step; unless the instructional strategies and materials embodied 


'S The definitions and scales underlying “emphasizing related professional development” and “receiving a minimum 
level of related district assistance” are described in the appendix. 
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in the curriculum are put into practice, however, the curriculum is not likely to be effective. The 
case studies indicate the importance of multiple sources of assistance (including professional 
development, on-site support, and other district or state assistance) to fully implement new 
curricula. 


Exhibit 13 


Of Continuously Identified Schools Engaged in Different Improvement Activities, 
Percent That Emphasized Related Professional Development or Received 
a Minimum Level of Related District Assistance 
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Exhibit reads: Among schools that continued to be identified for improvement in 2002-03 and reported 
implementing a new language arts program, 85 percent reported emphasizing related professional development or 
receiving a minimum level of related district assistance, 72 percent reported emphasizing related professional 
development; 59 percent reported receiving a minimum level of related district assistance; and. 43 percent reported 
emphasizing related professional development and receiving a minimum level of related district assistance. 


Note: See the appendix for sample sizes and additional statistical information. 


Source: TASSIE school survey. 


Many continuously identified schools adopted more than one new program and often 
layered them on top of existing programs, raising issues of how to cope with multiple new 
programs and how best to mesh the new with the old. Educators at identified schools were 
dealing with many new initiatives at the same time. In addition to learning to use data to inform 
instruction and aligning instruction with academic content standards, teachers were also faced 
with adopting more than one new curriculum simultaneously, often in addition to a school 
reform model. Thirty-two percent of schools that continued to be identified had adopted both 
new language arts and new mathematics programs in the last three years, and 48 percent of 
schools had been implementing either two new instructional programs (in mathematics and 


language arts) or a new instructional program (mathematics or language arts) in combination 
with a school reform model. 


Often, in the case study sites, new programs were added to existing programs rather than 
replacing them, unless it was a new district adoption or a program supported by a major grant. 
In some cases, mandated new programs conflicted with programs already in place. For example, 
schools in one urban district had been implementing Success For All for five years, when the 
district asked the schools to implement its new reading framework representing a different 
approach. The case studies also showed that some schools chose to forego grants to support new 
programs to avoid such conflicts. For example, two case study schools in different districts and 
states declined opportunities for Reading First grants because they were committed to other 
reading programs. 


The case study schools provided many examples of schools implementing multiple new 
core and supplemental programs at the same time. As Exhibit 14 illustrates, a school can be 
overburdened by multiple new initiatives that may run at cross-purposes. When schools try to 
launch many programs, they can lose focus and do a poor job with all of them, instead of 
focusing on a major strategy directed to the most important problem. Standards-based reform 
and NCLB intend to help focus school improvement efforts but, under pressure for solutions, 
districts and schools often attempt to solve problems by adding more and more programs. 


Exhibit 14 
Program Proliferation 


In the quest to help low-performing schools improve, an unintended consequence is a proliferation of 
instructional programs in schools, each with its own training and materials. One of several examples 
encountered in the case studies was a school in a large urban district which, under a Comprehensive 
School Reform Demonstration (CSRD) grant, had been implementing a self-paced skills training 
program in language arts and mathematics which the district was obligated to continue for two years in 
order to continue the grant. The school was also implementing the district-adopted curriculum in 
reading (Harcourt Brace) and in mathematics (Quest). In addition, the district had asked the school to 
choose between Success For All and Direct Instruction for next year—a requirement for all schools that 
might be designated Corrective Action. The school was also implementing Project READ, another 
district supported reading program and DEEP, a hands-on approach to mathematics. During the past 
year, third- and fourth-grade teachers also received InTech training for using computers in the 
classroom. 


Professional Development 


As part of providing technical assistance, districts are expected to help identified schools 
select and implement professional development. They do this through districtwide professional 
development sessions and through allocating resources to the school site including the provision 
of on-site assistance designed to support teacher development. 


Almost all districts provided identified schools with some type of professional 
development in 2002-03; however, the amount of that professional development, its 
relevance to instructional improvement, and its quality varied greatly across districts and 
even schools within districts. As Exhibit 15 illustrates, among districts with identified schools, 
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95 percent reported that they supported professional development in language arts instruction in 
identified schools. High percentages were also reported for aligning instruction with standards 
(91 percent), analyzing achievement data (84 percent), and mathematics instruction (80 percent). 


Exhibit 15 


District Reports of Types of Professional Development Provided to Identified Schools, 
Among Districts With Identified Schools 
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Exhibit reads: Among districts with identified schools, 95 percent reported that they supported professional 
development in reading/language arts instruction in identified schools. 


Note: See the appendix for sample sizes and additional statistical information. 


Source: TASSIE district survey. 


Principals of continuously identified schools reported similar figures: 95 percent reported 
receiving professional development in language arts instruction, 80 percent in analyzing 
achievement data, 79 percent in aligning curriculum with standards, and 74 percent in 
mathematics instruction. However, these figures do not reflect the amount of professional 
development, its relevance, or its quality. 


Among case study sites, districts differed in the amount and quality of professional 
development they provided as perceived by teachers. In one case study school, the district had 
assigned a full-time staff developer to the school with considerable expertise in mathematics 
curriculum and instruction. Her formal and informal professional development for teachers, 
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including after-school sessions and feedback in the classroom, was deemed highly valuable. In 
contrast, in another case study school, professional development on the school’s new reading 
curriculum was provided each Friday afternoon by two resource teachers who had no particular 
expertise in the curriculum. Teachers judged the sessions as poorly organized and ineffective. 
The case study schools also suggest a positive relationship between the intensity and quality of 
professional development and support for instructional change on the one hand, and teachers’ 
enthusiasm for and commitment to making changes. 


In general, additional full-time staff assigned to schools to support teacher development 
was a highly valued resource in the case study schools. Nationwide, only a third (33 percent) of 
districts with identified schools reported that such schools had been assigned such support staff. 
Although these staff play an important role in school improvement, fewer than a quarter (23 
percent) of continuously identified schools had been assigned full-time staff whose specific 
job was to support teacher development. 


These additional staff went by a variety of names—improvement facilitators, instructional 
support teachers, literacy coaches, mentor teachers—and assisted in a variety of ways to develop 
and implement school improvement plans and to support instructional improvement. They 
worked with school staff to collect and analyze student achievement data, implemented new 
curricula or instructional strategies, and organized teacher professional development (see Exhibit 
16 for examples from the case study sites). These additional school staff typically worked half- 
time or more in each identified school, with no classroom responsibilities of their own. 


Having extra school-based staff can help build the knowledge and skill of the faculty to 
improve instruction, but those extra staff need skills and knowledge of their own to carry out that 
role effectively. The case study schools varied in regard to how qualified and helpful they found 
the extra staff to be. In several examples, the staff designated to provide professional 
development focused on evaluating staff instead. In other cases, the skills of the additional staff 
were not well-matched to the needs of the school. For example, one large district that assigned 
curriculum facilitators to schools did so without regard to the specific needs of the teachers; 
hence, it was accidental whether the match was a good one. 


In case study schools, coaches or specialists, who provided regular, on-site professional 
development, were often paid out of Title I finds. This use of funds is consistent with NCLB’s 
requirement that schools identified for improvement under Title I must allocate 10 percent of 
their Title I budget to professional development. As was true in 2001-02, a majority but not all 
districts with identified schools reported requiring their identified schools to allocate a 
minimum percentage of their Title I budget to professional development. Seventy-nine 
percent of districts reported making this requirement; these districts serve 77 percent of the 
nation’s identified schools. 
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Exhibit 16 
Examples of Additional School-Based Staff to Support School Improvement 


A large urban district assigned central office curriculum specialists (most of whom were literacy 
specialists) to work full-time in low-performing schools, including schools identified for improvement 
under Title |. These improvement specialists helped principals and teachers disaggregate and analyze 
state and local achievement data, developed an annual instructional calendar that emphasized skills 
and content areas where students were weak, developed and administered formative school-based 
assessments, and reviewed the results of these assessments on weekly early-release days. Teachers 
praised the opportunity to collaborate with colleagues during this time in planning instruction. The 
specialists provided professional development to teachers both formally on the early-release days and 
informally as teachers requested it. 


In one mid-size suburban district all schools worked with a half-time literacy coach from a local college. 
Literacy coaches provided professional development for teachers, did classroom observations, 
modeled lessons, and met with teachers regularly. In one school that was identified for improvement 
under Title |, the literacy coach attended the upper and lower elementary team meetings, observed 
each classroom once every three weeks, and held voluntary grade level meetings before school. His 
major goals included making sure that teachers understood the new English and language arts 
standards and helping teachers learn to use running records and literacy circles. 


One large suburban district assigned all low-performing (including schools identified for improvement 
under Title 1) schools reading and mathematics support teachers. Support teachers were district 
employees who worked four days a week at a targeted school under the supervision of the district 
reading and math directors; on the fifth day they worked out of the central office. They helped to 
administer and score diagnostic and interim assessments, created and maintained a schoolwide 
database on student performance, participated in extended planning with grade level teams, mentored 
first and second year teachers (who were required to meet with the support teachers once a week), 
demonstrated lessons, and maintained libraries of instructional materials. Instructional support 
teachers relieved classroom teachers of much of the paperwork burden associated with scoring and 
tracking district assessment results. 


District and School Capacity to Support Improvement 


Districts vary enormously in their capacity to design and implement interventions and 
support school improvement efforts. Some schools also require greater levels of support and 
assistance than do others. Some of this variation can be explained by local context (e.g., the level 
of need and the proportion of low-performing schools and children to be served); some can be 
explained by the number of administrators and staff in central offices and schools. 


In 2002-03, large and urban districts were more likely to provide assistance of various kinds 
to identified schools; small districts—serving roughly a third of all identified schools—provided 
many kinds of assistance less often. Support for planning and data use—a key district 
responsibility under NCLB—was more likely to include a wider array of activities in larger 
districts and urban districts than in smaller districts and rural districts. For example, 71 percent of 
very large districts reported that they provided an extensive range of support for planning and data 
use (meaning that they provide all 10 of the types of support included in the TASSIE survey),'° 
compared with 50 to 52 percent of large and medium districts, and 37 percent of small districts. 


'° See the appendix regarding the 10 types of support. 
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Larger districts and urban or suburban districts were also more likely to report that they 
provided identified schools with assistance from school support teams, distinguished teachers, or 
principal coaches. For example, 77 percent of very large districts reported that identified schools 
received help from school support teams, compared with 43 percent of small districts (see 
Exhibit 17). Forty percent of rural districts with identified schools provided those schools with 
assistance from school support teams, compared with 66 and 68 percent of suburban and urban 
districts. Overall, only half of small and rural districts with identified schools provided any 
of these forms of assistance (school support teams, distinguished teachers, or principal 
mentors) in 2002-03. 


Exhibit 17 
Assistance Provided by the District to Identified Schools, by District Size 
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Exhibit Reads: Among districts with identified schools, 43 percent of small districts, 57 percent of medium sized 
districts, 67 percent of large districts, and 77 percent of very large districts provided school support teams to 
identified schools. 


Note: See the appendix for sample sizes and additional statistical information. 


Source: TASSIE district survey. 


Larger districts and urban or suburban districts were more likely to assign additional school- 
based staff to identified schools to support teacher professional development, and they provided 
greater support for professional development more generally. For example, nearly two-thirds (64 
percent) of very large districts provided identified schools with additional full-time staff to 


support teacher professional development. This practice was far less common among small 
districts, with only one-fifth of such districts reporting that they assigned additional staff (see 
Exhibit 17). Similarly, about half of urban and suburban districts assigned additional full-time 
staff to identified schools for teacher professional development, compared with just 15 percent of 
rural districts. Larger districts and urban districts were also more likely to report that identified 
schools received additional professional development or greater access to professional 
development resources. 


Finally, larger districts and urban districts were more likely to centralize decisions about 
curriculum and to support the alignment of curriculum with academic content standards and 
assessments. For example, larger districts were more likely to have required schools to adopt new 
reading and mathematics curricula than smaller districts (72 percent of very large districts had a 
required reading curriculum, compared with 53 percent of small districts). Larger districts also 
tended to provide schools with a greater range of support for curriculum alignment. For example, 
more than half (57 percent) of very large districts provided an extensive range of support for 
curriculum alignment, including local content standards, detailed curriculum guides, model lesson 
plans, and other tools to improve alignment. (Districts providing an “extensive” range of support 
engaged in at least 9 of the 10 activities identified in the TASSIE district survey.)'’ Only one- 
quarter of small districts with identified schools provided the same extensive range of support for 
curriculum alignment. The same pattern also held true for urban districts, compared with 
suburban and rural districts. 


Not all districts had the staffing necessary to provide assistance to their identified 
schools; this was particularly true in small districts. Nationwide, nearly half of districts with 
identified schools (45 percent) reported that they employed no staff who were specially charged 
with providing support for teacher development in identified schools. Smaller districts were less 
likely to employ staff with special responsibility for providing assistance to identified schools, 
and when they did, they tended to have fewer of these staff per identified school than larger 
districts. Larger districts had more of these staff available to provide assistance. This was true 
both in absolute terms (because larger districts had larger central offices and employed more 
staff) and relative to the number of identified schools in the district. For example, two-thirds of 
very large districts employed more than one full-time-equivalent (FTE) staff member per 
identified school to provide assistance to those schools, compared with one-third of small 
districts (see Exhibit 17). Only 6 percent of very large districts reported that they employed no 
special staff at all, compared with 27 percent of large districts, 38 percent of medium districts, 
and 55 percent of small districts. 


Survey data also showed that districts with more FTE staff available per identified school 
were more likely to have provided those schools with distinguished teachers, principal mentors, 
additional professional development, and school-based teacher development. These patterns 
suggest that adequate staffing is a key element of district capacity to provide assistance to low- 
performing schools. 


The case study sites illustrated the range of staffing resources available to different types of 
districts. At one end of the spectrum, an affluent suburban county system was able to assign a 


'7 See the appendix regarding the 10 support activities. 
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full-time professional development teacher, a full-time reading specialist, and a mentor teacher 
to each school in the county. In addition, the Title I office assigned an instructional specialist to 
each Title I school. At the other end of the spectrum, a small rural district had four central office 
staff members in total and no reading resource teachers in any of its schools. This district and 
several other case study districts like it relied exclusively on state and federal funding streams 
for needed resources for schools. 


Some districts had reorganized their central offices to free staff to work with low- 
performing schools. As described above in Exhibit 16, one large urban district had reassigned 
central office curriculum staff to work full-time in schools as improvement facilitators. In 
another large urban district, Title I coordinators based in the central office worked as school 
improvement team leaders in one or two schools each. Other districts had reorganized their 
central office staff into teams to oversee school improvement efforts. 


Districts often relied on special funding streams to provide essential support for 
additional staff positions, although special federal funds reached a relatively small 
proportion of schools that continued to be identified for improvement. In case study schools 
where new staff were assigned to identified schools to provide support, funding for those 
positions frequently came from federal sources, such as the Reading First, Title I, and the 
Comprehensive School Reform (CSR) program, or from state school improvement funds. 

Rarely were these positions funded locally. In a small district, for example, Reading First funds 
paid for the services of a literacy coach, the only additional support position in the school. In 
Louisiana, identified schools were encouraged to apply for CSR grants, in part because those 
grants bring additional staff and resources to schools. 


Although federal funding streams were an important resource for some schools and 
districts, they reached only a minority of all identified schools nationwide. Fourteen percent of 
schools that continued to be identified for improvement in 2002-03 received support through the 
Reading First program. Similarly, 17 percent of all continuously identified schools reported that 
they had received funds through the CSR program. 


The amount and intensity of assistance provided by districts depended both on the 
staffing and other resources available to the district and on the number of schools to be 
served. Of the estimated 1,800 districts that had identified schools nationwide in 2002-03, 
nearly three-quarters (73 percent) had just one or two schools identified for improvement in 
2002-03 (see Exhibit 3 in the introduction to this report). The majority of these districts, 
however, were small, with no more than three Title I schools overall—the districts least likely to 
have adequate staffing and other resources available to meet the needs of even the small number 
of Title I identified schools they were required to serve. In fact, the smallest districts in the case 
study sample—with the smallest numbers of identified schools—had few staff and very few 
options for mounting meaningful improvement efforts. These districts depended most heavily 
on their states and on outside sources of assistance to serve their identified schools. By contrast, 
large districts in the case study sample had the necessary staff and expertise in their central 
offices to organize effective support to schools, and where the number of schools to be helped 
was relatively small, these districts had the option of concentrating resources where they were 
needed most. As a result, large case study districts with a small fraction of all their schools that 
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were chronically low-performing were able to mount relatively intensive assistance efforts. In 
other large districts, local resources could be stretched thin because of overwhelming need. In 
one urban district, for example, all 11 of the middle schools in the district were identified for 
improvement. The district was investing substantial resources in turning those schools around, 
although the challenge of producing improvements in student achievement in 11 middle schools 
simultaneously was a formidable one. 


Schools differed in their organizational structures and capacity and therefore in their 
ability to launch effective improvement efforts and to take advantage of assistance and 
intervention. The case studies suggest that schools also need a minimal organizational structure 
in place to be able to benefit from assistance that is provided. Several schools had new 
principals—new to the school and new to the job. Others had experienced principals who 
simply were not strong leaders as judged by the lack of coherence and direction in the school’s 
improvement efforts. Similarly, schools varied in the proportion of inexperienced teachers on 
their staff and the rate of faculty turnover. Analogous to districts, schools with a small 
proportion of students achieving below grade level were better able to concentrate on those 
students, compared with schools in which virtually all students are low-performing. These 
differences among schools often reflected quite different needs in terms of the assistance and 
support required for principals and teachers. 


District Strategies for Targeting Assistance and Support 


NCLB assumes that states and districts will provide schools identified for improvement 
under Title I with the resources and assistance needed to improve instruction to make AYP and 
exit school improvement status. States and districts can respond to these provisions in NCLB by 
organizing assistance that is specially targeted to identified Title I schools or by ensuring that 
identified Title I schools are included in assistance efforts targeted to low-performing schools 
more generally. 


As shown earlier, most districts in 2002-03 were actively engaged in providing support of 
various kinds to identified Title I schools. The majority of districts with identified Title I schools 
(59 percent) were small, with enrollments under 3,500. Nearly a third of districts with identified 
schools (31 percent) had three or fewer schools in the district (Title I and non-Title I). These 
districts had little need to consider options for targeting assistance to their Title I identified 
schools, and little reason to differentiate the assistance they provided identified schools from the 
assistance they provided to others. Larger districts that served identified schools, schools that 
were low-performing but not identified, and higher-performing schools had to make choices 
about which schools to serve and whether identified Title I schools would receive more or 
different kinds of assistance. The analyses presented in this section include only districts with 
each of these three types of schools (36 percent of districts with identified schools), and so 
exclude a large number of smaller districts where all schools are low-performing or identified for 
improvement. 


Districts typically provided most kinds of assistance to all of their schools, whether or 
not they were low-performing or Title I identified. Most districts with low-performing 
schools, of which Title I identified schools are a subset, and other higher-performing schools 
provided most forms of assistance to all their schools, including assistance with planning, 
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analyzing budgets, and implementing research-based improvement strategies. Only a small 
percent of these districts provided such assistance only to identified Title I schools. For 
example, almost all of these districts identified research-based strategies for all their schools. 
Similarly, nearly three-quarters of these districts helped all types of schools—including those 
that were higher performing—write a school improvement plan. A small number of these 
districts—13 percent—helped identified Title I schools write or revise improvement plans, but 
did not provide that assistance to other kinds of schools. Similar ratios held for other types of 
support (see Exhibit 18). 


Almost every district had the same reading and mathematics curriculum 
requirements for all schools in the district, whether or not they were higher-performing, 
low-performing, or identified for improvement under Title I. Fewer than 5 percent of 
districts that had adopted programs reported that they had adopted a new curriculum or set of 
instructional materials in reading or mathematics for their Title I identified schools only. This 
may explain why principals of continuously identified schools cited reasons of instructional 
consistency, not Title I identification, for adopting new reading and language arts curricula. 


An exception lies in the area of principal mentors or coaches. A majority of districts with 
identified schools, other low-performing schools, and higher-performing schools that assigned 
principal mentors or coaches had restricted them to Title I or low-performing schools, rather than 
making them available to all types of schools in the district. Similarly, among districts that 
required the adoption of school reform models for at least some of their schools, about a third 
targeted that requirement to their low-performing and Title I identified schools (see Exhibit 18). 


Districts with low-performing schools, none of which were identified for improvement 
in 2002-03, offered the same kinds of support to those schools as districts with identified 
schools. In most respects, districts with identified Title I schools that were required respond to 
NCLB provisions regarding school support did not appear to act differently from other districts 
with schools they classified as low-performing but that were not identified for improvement. For 
example, districts with low-performing schools, none of which were identified under Title I, 
were just as likely to provide those schools with school support teams as districts with identified 
schools. The same held true for distinguished teachers and principal mentors, as well as for 
various kinds of support for planning. NCLB specifically requires that districts help identified 
schools develop an improvement plan, review the school budget, and identify research-based 
improvement strategies. Districts with low-performing schools, but without Title I identified 
schools, undertook these steps as often as districts with identified schools did. 


Districts provided the same kinds of assistance to schools in corrective action as they 
did to schools that had been identified more recently for improvement. Districts with both 
types of schools provided all types of assistance to schools identified for just one or two years 
just as often as they did to schools identified for three years or more. This pattern held true for 
all types of on-site assistance, additional staffing in schools, professional development, and 
support for planning and data use. 
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Exhibit 18 


District Support to Identified Schools, Other Low-Performing Schools, and Higher- 
Performing Schools, Among Districts That Had All Three Types of Schools 


To all types of 
schools, including 
higher-performing 

schools, low- 

performing 
schools, and Title | 
identified schools 


To all types of low- 
performing 
schools, including 
Title | identified 
schools, only 


To Title | identified 
schools only 


Identify research-based improvement 


F 87 7 2 
strategies 
Write or revise a school improvement 74 8 13 
plan 
School reform model adoption, among 
districts that require at least some of 66 11 22 
their schools to adopt a model 
Analyze and revise the school’s budget 
so that school resources are effectively 56 13 9 
allocated 
School support team 45 9 9 
Extensive range of support for planning 25 45 41 
and data use 
Extensive range of support for 24 4 4 
professional development 
Distinguished Teacher 20 3 6 
Principal mentor or coach 10 22 12 


Exhibit reads: Among districts with Title I identified schools, other low-performing schools, and higher-performing 
schools, 87 percent of districts identified research-based improvement strategies for all three types of schools. 
Seven percent of districts reported that they provided this form of assistance only to low-performing schools, 
including Title I identified schools. Two percent of districts reported that they provided this kind of assistance only 


to schools identified under Title I. 


Note: See the appendix for sample sizes and additional statistical information. 


Source: TASSIE district survey. 


Some districts provided more intensive assistance to schools identified for 
improvement for three years or more than to those identified for one or two years. For 
example, nearly half of districts (46 percent) reported that schools identified for three years or 
more received more intensive support from school support teams or other technical assistance 
providers. The same number of districts reported that these schools received larger school 
improvement grants. Half of districts reported that schools identified for three years or more 
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received greater monitoring and oversight from the district, compared with other schools 
identified for improvement. 


Districts with identified schools targeted schools for assistance and interventions on 
the basis of considerations other than or in addition to Title I program improvement status. 
Although considerable improvement activity was under way in schools and among districts, 
much of that activity in 2002-03 was occurring as a result of state and local accountability 
systems. Across the case study sites, district strategies for providing assistance included: 
providing the same services to all schools, targeting all low-performing schools, targeting all 
Title I schools, or targeting a particular subset of “problem” schools. In general, districts 
provided many supports to all their schools. In some of the case study sites, districts argued the 
need to help all schools because all were low-performing and therefore in need of assistance. In 
other cases, in which low-performing schools were a subset of all schools, districts provided 
extra assistance to all their low-performing schools. 


In addition to singling out high-need schools for assistance, several case study districts 
provided “preemptive” assistance to all of their Title I schools, including those that had not yet 
been identified for improvement, aiming to ensure that these schools did not lapse into 
improvement status. Case study districts that targeted assistance to all low-performing schools 
explained that because these schools moved in and out of official Title I identification, those that 
had moved out continued to need assistance. 


Districts could encounter problems if they provided some schools, but not others, with 
resources and assistance. For example, one suburban district with many high-achieving schools 
formed a School Improvement Unit to provide additional resources and oversight for the 
district’s lowest-performing schools. The districts’ well-regarded mathematics and reading 
supervisors and other central office staff worked only with schools in the School Improvement 
Unit, prompting complaints from parents at other schools in the district. The district director of 
instruction said, “It’s a challenge to reallocate the time and attention of central office staff. The 
community doesn’t want higher-performing schools to suffer.” Similarly, a large urban district 
retreated from its major reallocation of central office resources to low-performing schools as a 
result of pressure from the communities of the other schools. 


Public Reporting 


Another central principle underlying the Title I accountability system is that parents and 
the public have a clear understanding of the progress for which schools are being held 
accountable—the progress of school improvement efforts and the progress toward achieving the 
goal of all students reaching proficiency. NCLB has strengthened the reporting provisions of 
Title I by specifying that states and districts receiving Title I funds must issue “report cards” 
with state assessment results and lists of schools identified for improvement. NCLB also 
requires that state assessment results for student subgroups and the Title I improvement status of 
a school be reported, along with information about teacher quality. State report cards provide 
information about the performance of districts and states as a whole, whereas district report cards 
provide information down to the school level. NLCB identified the requirements for state and 
district report cards. ED published final guidance on how states and districts should prepare 
their annual report cards in September 2003 (ED 2003a). 
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Although most districts made school report cards available to the public in 2002-03, 
the number of districts that did so had declined. In 2002-03, 81 percent of districts made 
school report cards available to the public, down from 93 percent in the previous school year. 
Possible reasons for this decline may have been due to states’ and districts’ transitioning to new 
reporting formats as seen in case study sites. For example: 


¢ Louisiana did not provide school report cards during 2002-03—1in contrast with 
previous years when two types of school report cards were issued—because the state 
was revising the format of these report cards to comply with NCLB requirements by 
2003-04. (School performance data were available in other formats.) 


¢ Michigan no longer generated school report cards, instead leaving the preparation of 
school report cards up to districts, which had varied capacity in 2002-03 to generate 
data in the form required by NCLB (the SEA posted a checklist on its Web site with the 
items that schools and districts had to include in their public reports). 


¢ Maryland and Washington revamped their Web sites to provide 2002-03 school report 
card data, but not all of the data required by NCLB was readily available. Some 
districts in these states had not used the data available on the state Web site to update 
school report card information for 2002-03, whereas others made data available on their 
district Web sites. 


School report cards were most likely to include assessment data, but those data were 
not always disaggregated by all of the subgroups identified in NCLB. Information on (1) 
student performance data on statewide academic assessments, and (2) student performance at the 
school on state assessments compared with all students in the district and state were provided in 
school report cards in 97 percent and 89 percent of districts, respectively. The percentages were 
very similar for districts with Title I schools identified for improvement—98 percent and 88 
percent, respectively. 


In 2002-03, more than 60 percent of districts that prepared school report cards reported 
school assessment data for student subgroups for five of the six categories specified in 
NCLB—between 62 and 68 percent of districts with a subgroup enrollment of 10 percent or 
more (see Exhibit 19).'* Frequently, states and districts did not include assessment information 
about migrant students in their school report cards as is now required under NCLB (only 50 
percent of districts did). Little change occurred in public reporting for student subgroups, 
compared with 2001-02 (5 percent or less) except for migrant status (18 percent more). 


'S A threshold of 10 percent per subgroup was established for analysis to exclude districts with very small subgroup 
populations (i.e., districts where the subgroup population was too small to be included on report cards). This 
threshold does not necessarily reflect the district’s actual minimum number of students per subgroup for reporting 
purposes as defined in each state accountability plan. 
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Exhibit 19 


Whether Districts Disaggregated Student Assessment Data on 
School Report Cards, Among Districts With School Report Cards 
and a Subgroup Enrollment of 10 Percent or More 


Percent of districts 
By racial/ethnic group 68 
By limited-English-proficient status 63 
By special education status 65 
By economically disadvantaged status 62 
By gender 63 
By migrant status 50 


Exhibit reads: Among Title I districts that enrolled 10 percent or more racial or ethnic 
groups and produced school report cards, 68 percent disaggregated student assessment 
data by racial or ethnic group. 


Source: TASSIE district survey. 


The U.S. Department of Education’s nonregulatory guidance on report cards under Title I 
of NCLB specifies that individual school report cards are not required, but information about 
each school must be included in the district report card (ED 2003a). Education Week reported in 
its Quality Counts 2003 issue that of the 47 states and the District of Columbia requiring school 
report cards, 19 did not include or require disaggregated student performance data. (Two 
additional states made disaggregated data available on the Web or in other published reports; one 
state required school report cards beginning in 2003-04.) A review of state Web sites by the 
study team indicated that 37 states made some form of disaggregated student assessment data 
available online for the 2002-03 state report card which also suggest that some states were 
providing these data for schools in their state. 


NCLB calls for districts to provide information on student performance on the state’s 
additional academic indicators used in making AYP determinations. At the secondary level this 
includes the graduation rate for high schools. In 2002-03, among Title I districts that prepared 
school report cards, 81 percent included information on high school graduation rates (the 
percentage was the same for districts with identified Title I schools). 


New reporting requirements under NCLB were being met less frequently than data 
that had traditionally been reported. Although most states and districts had been providing 
information on school and student achievement and attendance rates to the public for years, 
some of this data did not meet NCLB reporting requirements (e.g., the method used to calculate 
high school graduation rates). Other information, such as school improvement status and teacher 
quality required under NCLB, was often not reported previously. In 2002-03, only 35 percent of 
Title I districts that prepared school report cards (42 percent of districts with identified Title I 
schools) included information about Title I improvement status in their report cards as required; 
in 2001-02, 31 percent of districts that prepared school report cards did so (47 percent of districts 
with identified Title I schools). Yet schools and districts often used other means to communicate 
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this information, such as letters to parents, parent information sessions, school newsletters, and 
state and district Web sites. District survey data indicated that among districts with identified 
schools, 94 percent sent written notification to the homes of parents that their child’s school had 
been identified for improvement. In some cases, states imposed additional requirements 
regarding parent notification of schools identified for improvement. For example, Arizona’s 
state accountability system required districts to meet with parents about school status. 


Education Week in its “Quality Counts 2003” issue also reported that 22 states required 
that school or district report cards include information about teacher characteristics, such as the 
percent with emergency credentials. Just four states publicly reported teacher qualifications 
disaggregated by school type in 2002-03. Only California, Indiana, Kentucky, Louisiana, and 
Tennessee provided parents with information about the credentials of every public school 
teacher on a Web site. TASSIE survey data indicated that 43 percent of districts included 
information on the percentage of teachers teaching with an emergency credential or provisional 
credential and 32 percent include information on the percentage of classes taught by highly 
qualified teachers in their 2002-03 school report cards. 
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Ill Public School Choice and Supplemental Services Under Title | 


In addition to keeping parents informed about the performance of their child’s school, 
NCLB calls for districts to provide parents with children in schools identified for improvement 
under Title I with additional options. Under NCLB, for Title I schools that do not make AYP for 
two consecutive years (i.e., “identified schools’), districts must provide parents of students in 
these schools with the option of transferring their children to another public school in the district 
that has not been identified for improvement. Ifa school does not make AYP for three years 
(one year after being identified), low-income parents of children in these schools also must be 
provided with the option of requesting supplemental educational services from state-approved 
providers for their child. NCLB requires that districts make available an amount equal to 20 
percent of the district’s Title I, Part A, allocation to fund Title I choice-related transportation and 
supplemental services. Unlike public school choice, supplemental services are offered only to 
children from low-income families and not to parents of all children in identified Title I schools. 
Parents are also to be notified about these options before the school year begins so that they have 
time to make informed decisions. Because Title I school choice and supplemental services were 
new requirements for most districts in 2002-03 (though some more limited Title I choice 
requirements applied to certain schools in 1999-2000), implementation for most districts in 
2002-03 represented their initial efforts to implement these requirements. 


In 2002-03, few parents transferred their child to schools not identified for 
improvement or received supplemental services and few districts were required to provide 
these options. In 2002-03, 16 percent of districts (1,800 LEAs) nationwide had 6,000 Title I 
schools identified for improvement and corrective action that were therefore required to provide 
Title I choice; 11 percent of districts (1,100 LEAs) had Title I schools in their second year of 
improvement and were required to provide supplemental services (see Exhibit 20). About 7 
percent of students in districts that offered supplemental services received services in 2002- 
03, whereas 1 percent of students in districts that offered choice transferred to another 
school. A number of factors emerged from parent focus groups that may have contributed to 
low participation in Title I choice and supplemental services, including parents not wanting to 
move their children if they were doing well in the identified school, late and inadequate parental 
notification, and transportation issues. 


Districts did not report challenges associated with the procedural requirements to 
implement choice and supplemental services to be major barriers to implementation in 2002-03, 
although they did report other challenges whose magnitude varied significantly by district type. 
For example, half of all districts with eligible students cited lack of providers in the area as a 
major challenge to implementing supplemental services and almost half of small districts with 
eligible students cited no alternate schools as a serious challenge to implementing Title I choice. 
Districts that did not provide parents with children in identified schools with choice or 
supplemental services were predominantly small, rural, and poor. Their ability to meet these 
requirements are challenged by the long standing issues that these types of districts have faced 
(e.g., limited numbers of schools, long distances between schools and neighboring districts, 
limited access to outside services). 
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These findings are elaborated on below in three parts: public school choice, supplemental 
services, and communication with parents. 


Exhibit 20 


Participation in Title | Choice and Supplemental Services in 2002-03, 
Among Districts With Identified Schools 


School choice Supplemental services 
Districts: 
Number required to offer option 1,800 1,100 
Number where option offered 1,200 500 
Percent were option offered 67% 48% 
Schools: 
Number where option required 6,000 1,300 
Number where option offered 5,100 800 
Percent where option offered 84% 58% 
Students (among districts that 
provided options): 
Number eligible 1,535,000 592,000 
Number that participated 18,000 42,000 
Percent that participated 1% 7% 


Exhibit reads: Among districts with identified Title I schools, an estimated 1,800 were required to offer school 
choice to students enrolled in identified schools and 1,100 districts were required to offer supplemental services 
to low-income students in schools identified for two or more years. The data presented in this exhibit are 
estimates at the 95 percent confidence interval with 32 degrees of freedom and estimates have been rounded. 


Note: The number of students eligible is underestimated because not all districts that should have provided 
choice and supplemental services reported providing these options and did not provide eligibility data. See 
Exhibits 21, 30 and 31 for explanations regarding the limitations of these data. 


Source: TASSIE district survey and Eligibility Dataset. 


Public School Choice Under Title | 


In 2002-03, 16 percent of all districts had identified Title I schools and therefore were 
required to provide public school choice, but only two-thirds (67 percent) of these districts 
did so. The districts providing choice accounted for about 5,100 identified schools or 84 percent 
of the 6,000 Title I schools identified for improvement in 2002-03. The percentage in 2002-03 is 
an increase over the 2001-02 rate when 54 percent of districts with identified schools offered 
some form of choice (note that all districts with identified Title I schools were not required to 
offer Title I choice in 2001-02). However, this percentage increase did not translate into more 
districts offering Title I choice in 2002-03 because Title I choice requirements in 2002-03 under 
NCLB are different from those in place in 2001-02 under IASA (see Exhibit 21). For Title I 
under IASA, Congress added requirements in 1999 and 2000 that school choice be offered to 
students in certain Title I identified schools (i.e., districts with identified schools that also 
received school improvement reserve funds), but the choice options did not necessarily meet the 
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current criteria specified under NCLB (i.e., that parents be notified of choice options before the 
start of the school year and that transferring students be provided with transportation to non- 
identified schools). 


Exhibit 21 


Districts That Offered Choice and Students That Exercised Choice 
in 2001-02 and 2002-03 


2001-02 2002-03 
Districts: 
Number required to offer choice 2,300 1,800 
Number where choice offered 1,200 1,200 
Percent where choice offered 54% 67% 
Schools: 
Number where option required 9,000 6,000 
Number where choice offered Not available 5,100 
Percent where choice offered Not available 84% 
Students (among districts that provided 
choice): Not available 1,535,000 
Number eligible Not available 18,000 
Number that participated Not available 1% 
Percent that participated 


Exhibit reads: In 2001-02 there were an estimated 2,300 districts nationwide with identified Title I schools 
(this number may be overestimated because not all districts with identified Title I schools were required to 
offer choice under IASA as described in the text). There were an estimated 1,800 districts with identified 
Title I schools in 2002-03 that were required to offer public school choice to students. The data presented in 
this exhibit are estimates at the 95 percent confidence interval with 32 degrees of freedom and hence the 
numbers may have been higher or lower than indicated. 


Note: All numbers have been rounded. Percentages of districts and schools were calculated by dividing the 
number that provided choice by the number required to offer choice. The number of students eligible for 
choice is underestimated because a third (33 percent) of the 16 percent of districts required to offer choice 
reported that they did not offer choice and did not provide data on the number of eligible students. 
Additionally, the estimates of the number of students eligible and participating in choice in 2002-03 has a 95 
percent confidence interval of 932,000 to 2,138,000 eligible students and a 95 percent confidence interval of 
9,000 to 28,000 students participating in choice. See the appendix for sample sizes and additional statistical 
information. 


Source: TASSIE district survey and Eligibility Dataset. 


Districts required to implement choice under NCLB, but that did not do so in 2002-03, 
were predominantly small, rural, and poor. This finding reflects, in part, that the majority (over 
half, see Exhibit 4) of districts with identified Title I schools were small rural districts and, in 
part, the limitations on choice in small rural districts. 


Some of the lack of implementation of Title I choice may have been due to lack of 


clarity at the local level about which schools choice options for students were required. For 
example, districts in one state were advised not to apply interventions, including choice and 
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supplemental services, if it appeared an identified school would make AYP in 2002-03 (based on 
2001-02 test scores). Some schools in these states that offered choice in 2001-02 continued to do 
so in 2002-03; schools that appeared as if they might make AYP for a second consecutive year 
did not. In another state, uncertainty over how to implement Title I school choice in schools in 
districts operating under desegregation orders complicated implementation. For instance, many 
districts in Louisiana delayed implementing choice provisions (including parent notification) 
pending discussions with the U.S. Department of Education over issues related to desegregation 
orders that affected more than 70 percent of the districts. At the same time, in 2002-03, 
Louisiana schools that continued to miss their growth targets after having been identified for 
improvement had to provide school choice options to parents under the state accountability 
system. 


The most common choice option available to parents of children enrolled in Title I 
schools identified for improvement was transferring to any other school in the district at 
the appropriate grade level that was not identified for improvement (see Exhibit 22), but the 
options varied significantly on the basis of district size and locale. Small (71 percent) and 
medium (72 percent) districts were more likely than others to respond that parents could choose 
among all schools in the district at the appropriate grade level, whereas urban (28 percent) and 
very large (50 percent) districts were more likely to have offered parents a subset of schools that 
had been paired with a sending (identified) school. As seen in large case study districts, the need 
to minimize transportation logistics played a part in explaining these differences (see example 
below). 


Exhibit 22 


School Choice Options Available to Parents of Children Enrolled in 
Identified Title | Schools, Among Districts That Offered Choice 


School choice options provided Percent of districts with identified schools 


All other schools in the district at the appropriate grade 


level that are not identified for improvement under Title |* oe 
A subset of schools that have been paired with the 19 
sending school* 

All other schools within a certain geographic zone 15 


Public schools outside the district 


Exhibit reads: Among districts with identified schools that offered Title I choice in 2002-03, 62 percent offered, as 
a school choice option, all other schools in the district at the appropriate grade level that were not identified for 
improvement under Title I. 


Note: As noted in the text, there were significant differences by district size for the two options with an asterisk (*); 
there was also a significant difference by urbanicity for the option of “a subset of receiving schools that were 
paired with a sending school.” See the appendix for sample sizes and additional statistical information. 


Source: TASSIE district survey. 


Among the 67 percent of districts with identified Title I schools that provided choice 
in 2002-03, there were an estimated 1.5 million students eligible to exercise choice because 
they were in an identified Title I school; about 2 percent of these students requested a 
transfer from an identified school and about 1 percent then transferred to another school. 
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Of the estimated 1.5 million eligible students, 28,000 requested a transfer to a non-identified 
school, and about 18,000 or 1 percent of eligible students actually transferred because their 
school was identified for improvement (Exhibit 21).'” 


The median number of eligible Title I students in a typical district offering choice in 
2002-03 was 500 (the average was 2,000). In 56 percent of districts with identified 
Title I schools where choice was offered, no students transferred. In the remaining 44 
percent of districts where choice was offered and some students elected to transfer 
schools, the median number of students that transferred was 10 (the average was 55). 


Similarly, 67 percent of identified Title I schools with students eligible to transfer (in 
districts where choice was offered) reported that no students transferred. In the 
remaining 33 percent of identified Title I schools where students elected to transfer 
schools, the median number of students that transferred to another school was eight (the 
average was five). 


There was a great deal of variability among districts offering school choice. So, for example, 20 
percent of districts had less than 100 students eligible to transfer and 30 percent of districts had 
1,000 or more eligible students. Exhibit 23 illustrates the distribution across districts in the 
number of students that actually transferred in 2002-03. 


Exhibit 23 


Distribution of Districts That Offered the Title | School Choice Option by Number 


of Participating Students, Among Districts With Identified Schools in 2002-03 


Number of students who transferred out of an identified Percentage of districts 
Title | school using the Title | school choice option: offering Title | choice 
None 56 
1 to 9 students 21 
10 to 99 students 19 
100 or more students 5 


Exhibit reads: Among districts with identified Title I schools that offered choice in 2002-03, 56 percent had 
no students transfer. 


Note: See the appendix for sample sizes and additional statistical information. 


Source: TASSIE district survey. 


'° The number of students eligible to exercise choice is underestimated because a third (33 percent) of the 16 percent 
of districts that had at least one Title I school identified for improvement indicated that they did not offer choice 
under Title I and did not provide this information. Any estimates of the number of students who requested a 
transfer and then transferred to non-identified schools include data only from districts that offered school choice 
to students in identified Title I schools. 
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The location (urbanicity) and size of a district was related to the number of students 
eligible for Title I choice; there were significant differences by district size and locale in the 
number of students eligible to exercise choice and the number of students who requested a 
transfer. As might have been expected, averages were highest in districts with the highest 
number of identified Title I schools—very large, urban districts (see Exhibit 24). Thus, for 
example, in two very large urban case study districts, the number of students eligible for public 
school choice was about 12,000, contrasted with a very large suburban case study district and a 
small rural case study district both with around 300 students eligible for choice. 


Exhibit 24 


Average Number of Students Eligible to Exercise Choice, Among Districts 
With Identified Schools and That Offered Title | Choice 


All District size 
districts Small Medium Large Very large 
2,200 500 600 1,900 15,000 
District urbanicity 
Urban Suburban Rural 
2,200 6,200 1,100 500 


Exhibit reads: Among districts with identified Title I schools that offered choice, the average 
number of students eligible to exercise school choice in 2002-03 was 2,200. In small districts 
with identified schools that offered Title I choice, the average number of students was 500; 
the average number was 600 students in medium districts; 1,900 in large districts; and 15,000 
in very large districts. 


Note: Numbers rounded to the nearest hundred. See the appendix for sample sizes and 
additional statistical information. 


Source: TASSIE district survey. 


The number of students that requested a transfer to a non-identified school may 
reflect the many considerations that go into a parent’s choice of schools for their child such 
as how well their child is achieving, the quality of the school staff, the availability of special 
programs, and the proximity to their home, as described in parent focus groups. Late and 
inadequate parent notification and limited choices for alternate schools, described later in this 
section, may have also contributed to low parent participation. In addition to the identification 
status of a school, research at the case study sites indicated that other factors influenced a 
parent’s decision in selecting a school. 


* One parent in an identified school in a very large urban district indicated her reasons 
for not transferring her child to a non-identified school, even though her child’s school 
was identified: “I did my own research and knew about it [identification]...but I liked 
the principal and loved the kindergarten teacher. I like the K-3 program and that we 
have school uniforms. Plus, it’s the neighborhood school, my husband went here, and 
his father went here.” She added: “Test scores are not just [the] school’s 
responsibility—I’Il put in the effort too. It’s a well-rounded community, I’m very 
happy here and my child is doing amazing work.” 
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¢ Another parent explained her decision to stay as follows: “As a parent, you have the 
option to pull your kid out or to stay with the promise of school improvement and the 
transformation plan. You stay because this is your community.” 


¢ Some parents in a small rural district indicated that they had selected their child’s 
school because, as a small school with small class sizes, it afforded more personalized 
attention: “I can call our principal at home. It’s like that with the teacher also, if I need 
something I can call her at home.” In schools that serve limited English proficient 
(LEP) students, the availability of bilingual staff who can communicate with parents is 
a strong incentive for parents to keep their children at the school. 


¢ Proximity to home and ease of access were also important considerations. 


In districts with choice policies that predated NCLB, parents who had already exercised 
choice may have reduced the number of students exercising choice in 2002-03. Three case study 
states—Arizona, Michigan, and Washington—have open enrollment policies, and parents had 
already selected the school they wanted their child to attend.”” Unlike some other choice 
policies, NCLB requires that transportation be provided to students in identified Title I schools 
who choose to attend a non-identified school. 


Data from other sources also suggest that many parents did not immediately think of 
transferring their child when a school was identified for improvement in 2002-03. A national 
2003 poll of the public’s attitudes toward the public schools indicated that, for schools identified 
for improvement, 74 percent of parents preferred to have additional efforts made in their child’s 
school, 25 percent preferred to transfer their child to a non-identified school, and | percent had 
no opinion. The poll also showed that the public was divided in regard to whether parents in the 
community had enough information to choose another school for their children to attend if their 
current school was identified for improvement; 48 percent of public school parents believed 
parents in their community had enough information, and 50 percent believed they did not 
(2 percent did not know) (Lowell and Gallup 2003). 


At the same time, more than half of the districts (53 percent) that offered choice 
reported that no students received their first choice of schools when transferring to a non- 
identified school—a fact that may have influenced the final decision about whether to 
transfer to another school or not. Nationwide, of the students that requested a transfer in 
2002-03, an estimated 17,000 (or 62 percent of those requesting a transfer) received their first 
choice of schools. Differences by district size in the number of students who received their first 
choice of schools when transferring were also significant. Students in small districts were less 


°° NCLB choice requirements are among a growing number of options that provide parents with choices within the 
public education system. Regions and districts of different sizes vary to a considerable degree in the availability 
and type of public school choice programs they offer. Additionally, districts experienced in developing options 
may have been better prepared to carry out the NCLB choice requirements. An analysis by the National Center 
for Education Statistics, Trends in the Use of School Choice 1993 to 1999, showed that the decrease in assigned 
public school enrollment had almost completely been offset by an increase from 11 to 14 percent in parentally 
chosen public school enrollment (NCES 2003). TASSIE district survey data indicated that about half (49 
percent) of districts with and without identified schools offered some form of public school choice in 2002-03; 
this compares with 41 percent in 2001-02. 
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likely to have received their first choice of schools because there were fewer schools to choose 
from compared with medium and large districts. 


The number of non-identified schools available as alternatives can influence the 
extent to which parents choose to send their children, at all grade levels, to non-identified 
schools. The number of schools was also linked to the size of the district, but the availability of 
alternative schools at the middle and high school levels was limited across all districts (see 
Exhibit 25). For example, although having no alternate schools available at the elementary level 
was less likely, more than half of districts with identified schools reported they had no alternate 
high schools for parents to choose and over a third had no alternate middle schools. There were 
also significant differences by district size in the average number of alternate schools available to 
families in identified schools, with very large districts having the largest numbers of alternate 
schools (e.g., 17 at the elementary level). But having more alternate schools did not necessarily 
guarantee more choice as seen in some of the case study districts (see Exhibit 26). 


Exhibit 25 


Number of Alternate Schools Available to Parents With Students 
in Identified Title | Schools, Among Districts That Offered Choice 


Mean number of 
schools 
Alternate schools for children in elementary grades 3 
Alternate schools for children in middle grades 1 
Alternate schools for children in high school grades 0 


Exhibit reads: Among districts with identified schools and that offered choice in 2002-03, the 
mean number of alternate schools available to parents with students in identified Title I 
elementary schools was three. 


Note: See the appendix for sample sizes and additional statistical information. 


Source: TASSIE district survey. 


Exhibit 26 
School Choice in Districts With Few Alternate Schools 


A very large urban case study district had more than 11,000 students eligible for Title | choice in 37 
schools but had very few alternative schools to choose from. (At the middle school level, it had none 
because they were all identified for improvement.) To try to provide improved services to students in 
the lowest performing schools, the district drew on additional resources such as increased support staff 
and funding as an alternative. To minimize transportation logistics, the district limited choices by 
pairing identified schools with two to four non-identified schools from which parents could choose. Two 
other case study districts, one medium and one small, had only one alternate elementary school that 
was not identified for improvement. In one of these districts, the school available to serve as an 
alternate was quite small and in a rural area of the county. 
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Districts were taking steps to increase the options available to parents of children in 
identified schools. ED draft non-regulatory guidance that was available in 2002-03 stipulated 
that a district had the flexibility under NCLB to determine which schools, among those not 
identified for improvement, would comprise the range of alternates to which eligible students 
could transfer. However, Title I regulations (cited in the guidance) also noted that capacity 
constraints were not allowable reasons for not offering students the transfer options, although 
districts may take capacity into account when deciding which schools to offer as transfer options. 
In addition, the guidance suggested that in situations where no schools were available as choice 
options (e.g., only one school in the district), districts, to the extent practicable, must establish a 
cooperative agreement with other districts in the area or could offer supplemental educational 
services (ED 2002). Some districts (5 percent) added teachers or classrooms to schools within 
the district that had not been identified for improvement—a higher percentage of very large and 
urban districts (15 percent and 14 percent, respectively) took this approach. A few districts (4 
percent) attempted to negotiate agreements with neighboring districts, but they were not 
successful. 


* One medium size case study district tried to establish an agreement with a neighboring 
district, but the other district declined the request to place students in one of its non- 
identified schools. The first district was waiting to hear from other nearby districts 
about their willingness to accept its students. One reason for the lack of interest from 
neighboring districts was the policy the sending district had established: if parents 
opted to enroll their children at schools outside the district, the district “would rank 
students, giving priority [for transportation] to the lowest performing students.” 


¢ Other case study data indicated that districts were offering additional services to 
students in identified schools (e.g., before- and after-school programs, academic 
tutoring), particularly if the district was faced with difficulties in expanding capacity 
and where sending students to a neighboring district was not possible (1.e., the nearest 
district was too far away or would not serve students from another district). 


Similar examples were provided in another national study of NCLB implementation which 
included visits to 15 school districts in 2002-03. They reported that school choice had been 
difficult to implement in some districts because they had few or no potential receiving schools 
that could serve the right grade levels and were not in school improvement themselves. Districts 
had also met with only limited success in persuading neighboring districts to accept out-of- 
district transfer students. Some districts had to offer supplemental services to students instead of 
choice in schools in their first year of school improvement (Center on Education Policy 2003). 


The challenges to the successful implementation and use of public school choice for 
students in identified schools generally follows the same pattern in 2002-03 as that for 
2001-02; again, the magnitude of the problems associated with procedural requirements 
were generally not large, but some challenges varied significantly by district type. Overall, 
only a small portion of districts were greatly affected by these challenges, with difficulties in 
expanding capacity (39 percent) and lack of space (33 percent) identified most frequently (see 
Exhibit 27). Districts reported that issues such as no alternate schools in the district (72 percent), 
lack of transportation to alternate schools (76 percent), and inadequate funding for choice-related 
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transportation and supplemental services (86 percent) were generally not a problem or posed 
only a small problem in 2002-03. The extent to which these issues remain a minimal barrier to 
implementation may depend on the extent of growth in the number of identified schools within a 
district and in parent demand for both choice and supplemental services (e.g., high demand may 
exceed funding from the 20 percent set-aside). 


Exhibit 27 


Challenges Faced by Districts That Implemented School Choice, 
Among Districts Required to Offer Choice 


Great Moderate Not at all or 
Type of challenge extent extent small extent 
Expanding capacity 39 12 49 
Lack of space in alternate schools 33 22 45 
No alternate schools within the district 17 12 72 
Lack of transportation to alternate schools 14 10 76 
Inability to negotiate agreements with other 12 9 79 
districts to receive students who wish to transfer 
Inability to meet the needs of LEP students in 
10 13 77 
alternate schools 
Inability to meet the needs of special education 
; 9 16 75 
students in alternate schools 
An amount equal to 20% of the district’s Title | 
allocation is not adequate to provide 
transportation to all students who request a 9 6 86 
transfer to alternate schools or supplemental 
services 
Inadequate information for parents about school 
: 3 2 94 
choice options 
Inadequate information for parents about the 
status of their child’s school as identified for 3 3 94 
improvement under Title | 


Exhibit reads: Among districts that were required to offer choice and made choice options available in 2002-03, 
39 percent responded that expanding capacity was a challenge to a great extent, 12 percent responded that it 
was a challenge to a moderate extent, and 49 percent responded that it was not a challenge or a challenge to a 
small extent to implementing school choice. 


Note: Totals may not add to 100 percent because of rounding. See the appendix for sample sizes and additional 
statistical information. 


Source: TASSIE district survey. 


On the other hand, some differences in the severity of challenges by district size and locale 
were significant. For example, small districts (42 percent) were more likely than others to have 
responded that no alternate schools in the district presented a serious challenge to implementing 
choice; small districts (30 percent) were more likely than others to have responded that lack of 
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transportation to alternate schools presented a serious challenge; and suburban districts (44 
percent) were more likely to have responded that lack of space in alternate schools was a serious 
challenge. Small (88 percent) and rural (75 percent) districts made up the majority of districts 
required to implement choice but that did not do so in 2002-03. 


Supplemental Educational Services 


Under NCLB, states and school districts were required to implement the supplemental 
educational services provisions at the beginning of the 2002-03 school year. The non-regulatory 
guidance issued by the U.S. Department of Education in December 2002 provided states and 
districts with additional information about how to implement these provisions in 2002-03. 
Specifically, NCLB requires that supplemental services be consistent with the state’s academic 
content standards and with the instruction provided by the school district. In addition, services 
must be provided outside the regular school day. School districts, in consultation with parents 
and providers, must develop specific educational goals for each student. Supplemental service 
providers must measure the student’s progress regularly and report regularly on that progress to 
teachers and parents. Supplemental services may be provided by a variety of agencies, including 
for-profit and nonprofit entities, approved school districts, faith-based organizations, and 
approved public or private schools. Data gathered through the state level interviews, district and 
principal surveys and site visits to the sample of case study districts provided a look at the early 
implementation of supplemental services as defined by NCLB. 


Implementation of the supplemental services provision was generally slow and uneven 
in 2002-03. States were slow to publish lists of approved providers, districts were slow to 
organize services, and few parents signed their children up for additional supplemental 
educational services. 


State Actions 


Each state is required to develop criteria for selecting providers, maintain a list of providers 
and provide school districts with a list of approved providers in their geographic locations, 
monitor provider services, and withdraw approval from providers that do not meet the statutory 
requirements to increase students’ academic achievement in two years. 


Very few states gave a list of approved providers to districts before the beginning of 
the 2002-03 school year. Only five states published lists of approved providers before 
September 2002. By Jan. 2003, 21 more states posted lists and 41 states plus the District of 
Columbia posted lists by the end of June 2003. Most state education agencies (SEAs) launched 
their provider identification process in 2002-03, but some time was required to generate an 
adequate list of providers. For example, Maryland had approved two providers at the beginning 
of the 2002-03 school year, but had added 12 more by spring 2003. 


States have flexibility in developing a provider approval process. At the same time, NCLB 


calls for selection criteria that promote participation by the maximum number of providers to 
ensure that parents have as many choices as possible. Data from state sources (e.g., state 
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policies, state interviews) indicate that states with identified providers by April 20037! often 
employed a rubric or template to evaluate provider applications for 2002-03. The rubrics were 
based on the requirements established in NCLB and listed in the Department of Education’s non- 
regulatory guidance. Many states also used the criteria outlined in the Council of Chief State 
School Officers (CCSSO) SEA Toolkit on Supplemental Services in the NCLB Act of 2001, 
which in turn was based on NCLB criteria as well as additional measures (CCSSO 2002). 
Elements adopted from the Toolkit other than those specified in NCLB included: (1) evidence of 
highly qualified staff (with some states requiring criminal background checks as required by 
state law for teacher licensure or public school-related employment regulations); (2) letters of 
reference offering information about positive program results as part of a demonstrated record of 
effectiveness; (3) adequate organizational resources to meet consumer demand; and (4) 
conditional approval of newly developed programs. 


Finally, some states also added state-developed criteria that included: (1) supervision of 
services by a certified teacher; (2) evidence of highly qualified paraprofessionals; (3) evidence of 
interactions with students if the providers used a Web-based program or specific technology 
requirements; (4) a designated pricing structure; (5) a defined service area; and (6) independence 
from district or school resources (e.g., the district or school was not required to provide space or 
resources). 


In 2002-03, the number and services of state-approved providers did not adequately 
match local needs. As discussed later in this section, many districts cited lack of providers as a 
major challenge to implementation. By April 2003, 43 states and the District of Columbia had 
approved a total of 1,017 providers, for an average of 23 per state (the same number as in June 
2003); however, the approved providers were not evenly distributed across states. Thirty-one 
states had between one and 20 providers each, six states had between 21 and 30 providers, four 
states had between 45 and 100 providers, and two states had over 100 providers (Policy and 
Program Studies Service 2003 and state interviews). 


Among districts that provided supplemental services, the average number of supplemental 
service providers available to students by spring 2003 (April-May) was five per district (a 
median of three). 


Even though it may have appeared that there were providers statewide, many 
providers did not serve rural areas or districts with small numbers of students or students 
with special needs. In other areas, providers lacked the capacity to meet the demand for 
their services. A Title I director in a small rural case study district stated: “We don’t have 
Sylvan up here. There is not a single provider who wants to come up here because we don’t 
have enough students.” In addition, online providers had not been available to some rural 
communities where Internet access and computer equipment were limited or unavailable. In one 
high-poverty case study district that offered online services, parents needed to have computers 


*! Two states, Florida and Wyoming, had no identified schools in their second year of improvement and, as a result, 
were not required to develop a list of approved providers in 2002-03. Spring 2003 (April-May) was the study’s 
data collection time frame for state-level and provider information on supplemental services to coincide with 
district and school survey data collection. State-level policies and practices after this time period would have 
little impact on 2002-03 local-level implementation of supplemental services. 
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and Internet access to have used providers’ services, a luxury that many parents in the district 
could not afford. According to local respondents, without hands-on assistance, only higher 
functioning students had the discipline to participate in a distance-learning program, thereby 
excluding some of the students most in need—younger students, students whose English 
proficiency was limited, and those who were disabled.” 


In some cases, providers found themselves having to restructure their programs because 
they were unable to hire enough staff due to districts’ inability to provide sufficient notice 
regarding the number of students that signed up for services; others had to turn students away 
because of insufficient staff. For example, a large urban case study district had 6,000 eligible 
students, but the eight approved providers could serve only 1,300 students or 22 percent of those 
eligible. Some providers interviewed indicated that they needed to serve a minimum number of 
students for their businesses to be viable. Meeting that threshold was difficult for small districts 
(ED 2004b). 


Approved providers represented a variety of organizations. As of April 2003 (the 
period of survey data collection), the bulk of providers were private providers (e.g., not-for-profit 
private groups or for-profit organizations such as Sylvan Educational Solutions, Kaplan 
Educational Centers, Kumon Math and Reading Centers). A third (32 percent) of approved 
providers were school districts and schools (see Exhibit 28). The make up of providers remained 
similar through the end of the 2003-04 school year (June 2003). 


Exhibit 28 
Supplemental Services Providers in April 2003 
Number of Percent of 

Type of provider providers total 
Private: Faith-based 18 2 
Private: Online 98 10 
Other private 472 46 
Districts and public schools 326 32 
Colleges and universities 32 3 
Other or unknown types 71 7 


Exhibit reads: Among states that approved supplemental services providers, 18 were 
faith-based organizations or 2 percent of all approved providers as of April 2003. 


Note: Information on the types of approved providers was available from 43 states and the 
District of Columbia. Totals may not add to 100 percent because of rounding. See the 
appendix for sample sizes and additional statistical information. 


Source: Policy and Program Studies Service unpublished database and TASSIE state 
interviews. 


°° Tn cases where there are no approved providers available to supply supplemental services to disabled or limited 
English proficient students, the district is required to provide supplemental services, either directly or through a 
contract. 
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While states were focused on the process of approving providers for services in 2002- 
03, many states did not establish systems for monitoring provider performance or a process 
for withdrawing providers from an approved list.” Twenty-two states completed standards 
for monitoring providers and 20 states completed standards for withdrawing providers in 2002- 
03. Alternatively, 20 states that provided data on the development of standards for monitoring 
providers reported that they had not yet established systems for monitoring performance, and 23 
states reported that they were still developing a process for withdrawing approval of providers. 
Most of the states that had developed or were working on their monitoring process planned to 
rely on reports from one or multiple outside sources (e.g., districts, schools, providers, parents), 
and two states reported that they were using an outside entity (their regional education 
laboratory) to evaluate provider performance. 


¢ The majority of states that began addressing monitoring issues emphasized the 
gathering of student performance data (primarily on assessments) as part of their 
proposed monitoring process, but they also identified other sources of data to be 
collected (e.g., reports on attendance and the number of dropouts, parent and school or 
district satisfaction surveys). 


¢ In five states, the frequency of provider monitoring differed, depending on the 
provider’s approval status (e.g., full vs. conditional approval). 


¢ Of the 20 states that established standards for withdrawing provider approval, eleven 
states reported that they adopted only the criteria for withdrawal outlined in 
NCLB—failure to improve student achievement over a period of two years, failure to 
comply with eligibility requirements, and assurances and responsibilities of providers 
specified in NCLB. Another eight states reported that they were using student 
achievement data for a period of less than two years (e.g., for providers with only 
provisional approval), some form of parent and district satisfaction data, monitoring 
data, and evaluation data from forms completed by providers and districts. 


Case study states reflected the range of implementation that existed in 2002-03: Louisiana 
had developed a systematic approach to provider identification and monitoring (see the 
description in Exhibit 29), whereas Maryland and Arizona had yet to consider monitoring. 
Michigan had approved 24 providers and was planning to collect district summaries of student 
progress (however measured) and look at state assessment results; Washington formed a review 
committee that met periodically to review applications and to establish a monitoring tool. 


*> Data were not available for six states regarding standards for monitoring providers and for five states regarding 
the establishment of standards for withdrawal of provider approval (these states did not respond to the request for 
an interview). 
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Exhibit 29 
State Criteria for Approving and Monitoring Supplemental Service Providers 


Louisiana’s SEA hired an outside firm that had worked with CCSSO to develop its Toolkit to assist the 
SEA in developing the state’s supplemental services provider application and to train reviewers. In 
addition, district staff from districts with students eligible to receive services were interviewed to develop 
criteria for approving providers. The resulting Supplemental Education Services Model used three 
criteria against which the qualifications of providers were assessed: 


(1) Was there a diagnostic assessment, or an appropriate process, that would be used by the provider to 
identify student weaknesses and achievement gaps, as well as to measure gains in student 
achievement (the latter for the purposes of provider accountability)? The diagnostic tool had to be 
tied to the curriculum of the provider, which should in turn be tied to state content standards. 


(2) Was targeted remediation or instruction aimed at addressing the individual skill gaps revealed during 
the assessment and based on an individual learning plan? 


(3) Was there a post-assessment to determine whether or not student gains occurred and a plan either 
for reteaching skills or for identifying new skill sets for instruction? 


Prospective providers were asked to complete a three-step review process. The first step was to have 
them complete the provider application, which included the requirements outlined in NCLB (e.g., has a 
demonstrated record of effectiveness, is financially sound). In the second step, providers were asked to 
provide information related to the model’s three requirements—the providers’ diagnostic processes (how 
instruction would be individualized) and their methods for delivering services. Face-to-face interviews 
were also conducted to assess provider qualifications. 


The state was also working on designing a monitoring process for providers that went beyond self-report 
data. As part of the state’s larger effort to monitor after-school providers, the state had established an 
online database in January 2003 that providers used to submit data on each child receiving services. 
The database generated reports for the user as well as the state. The SEA was planning to add more 
information requirements to the existing system for supplemental service providers such as achievement 
data collected by providers. The data would allow the SEA to also look at a student’s progress on state 
tests. 


District Actions and Student Participation 


School districts are responsible for notifying parents of their children’s eligibility to receive 
supplemental services and for providing parents with adequate information to select providers for 
their children. Parents can select any approved provider in the area served by the school district 
or within a reasonable distance of the school district. School districts must make arrangements 
to reimburse providers directly for the services they provide to eligible students. 


Few districts were required to offer supplemental services in 2002-03; of those that 
were, roughly half did so. Only 11 percent of Title I districts nationwide, or approximately 
1,100 LEAs, had Title I schools in their second year or more of school improvement where 
students were eligible to receive supplemental services (see Exhibit 30). But just under half of 
the districts (48 percent or 500) that had identified schools required to offer supplemental 
services were implementing services by spring 2003 (these districts contained about 800 
identified schools or 58 percent of the Title I schools required to offer supplemental services). 
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Districts not providing supplemental services in 2002-03 were predominantly small and rural, 
with medium to high poverty. 


Exhibit 30 


Districts That Offered Title | Supplemental Services, 
Among Districts Required to Offer Services 


2002-03 

Districts: 

Number required to offer services 1,100 

Number where services offered 500 

Percent where services offered 48% 
Title | schools: 

Number where services required 1,300 

Number where services offered 800 

Percent where services offered 58% 


Exhibit reads: In 2002-03, there were an estimated 1,100 districts required to offer 
supplemental services (regardless of the provision of services) to low-income students 
because they had Title I schools identified for improvement for more than one year. The data 
presented in this exhibit are estimates at the 95 percent confidence interval with 32 degrees of 
freedom and hence the numbers may have been higher or lower than indicated. 


Note: The numbers are estimates based on district survey responses. Percentages were 
calculated by dividing the number required to offer services by the number offering services. 
See the appendix for sample sizes and additional statistical information. 


Source: TASSIE district survey. 


Nationwide, an estimated 1,300 identified Title I schools were required to offer 
supplemental services in 2002-03 (regardless of whether they actually provided services). The 
number of schools and students in 2002-03 affected by supplemental services requirements was 
not large either, whereas district size and location significantly influenced whether there were 
students eligible for services. As defined by respondents to the district survey, these are the 
Title I schools identified for two or more years. The median number of schools in a district with 
students eligible to receive services was one (the average was four), but there were significant 
differences by district size and location with very large and urban districts having the highest 
numbers of schools (13 and 7 schools, respectively). The median reflects the composition of 
districts with identified schools; that is, more than half (59 percent) of districts with schools 
identified for improvement were small and medium districts that averaged very few identified 
schools (e.g., the average number of Title I schools that had eligible students was two in medium 
districts compared with 13 in very large districts). 


Nationwide, roughly 7 percent of eligible students in districts that offered 
supplemental services received services from an approved provider. Among districts that 
reported they had schools required to offer supplemental services in 2002-03, there were an 
estimated 791,000 students from low-income families eligible to receive supplemental services. 
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In districts that actually offered services, an estimated 592,000 students were eligible and 42,000 
of those students (7 percent) received services from an approved provider (see Exhibit 31). 


Exhibit 31 


Students in Identified Schools Eligible for and Receiving Supplemental Services, 
Among Districts That Reported Having Identified Schools and Provided Services “ 


2002-03 
Estimated number of students eligible to receive services in districts 592.000 
providing services , 
Number of students that received services from an approved provider 
oe han te : 42,000 
in districts providing services 
Percent of eligible students 1% 


Exhibit reads: Among districts that offered services, an estimated 592,000 students were eligible to receive 
supplemental educational services in 2002-03 because their schools were identified for improvement under 
Title I for more than one year. The data presented in this exhibit are estimates based on a 95 percent 
confidence interval with 32 degrees of freedom and estimates have been rounded. 


Note: The percentage was calculated by dividing the number of students who received services by the number 
of students eligible for services in districts that provided services. The numbers are estimates based on 
districts that provided services. The estimates of the number of students eligible for and that received 
supplemental services in 2002-03 has a 95 percent confidence interval of 111,000 to 1,073,000 eligible 
students and a 95 percent confidence interval of 23,000 to 61,000 students that received services. The 
estimates of the number of students who received services are based on the 48 percent of districts with 
schools that were required to offer and provided supplemental services to students in identified Title I 
schools; this is also true of the estimated number of low-income students eligible to receive supplemental 
services in districts offering these services (see footnote below regarding these estimates). See the appendix 
for sample sizes and additional statistical information. 


Source: TASSIE district survey. 


¢ The median number of students eligible to receive services per district that provided 
supplemental services was 500 (the average was 2,400). In 40 percent of the districts 
with identified Title I schools where supplemental services were offered, no students 
received services. In the remaining 60 percent of districts with identified Title I 
schools where some students received services during the 2002-03 school year, the 
median number that received supplemental services was 80 students (the average was 
200). 


** The 791,000 students eligible to receive supplemental services included students in all districts with Title I 
schools identified for two or more years that reported they had Title I schools required to offer supplemental 
services to students, regardless of whether or not the district offered services. The numbers are underestimated 
for all students eligible for services because 46 percent of districts required to offer supplemental services 
reported they did not have schools required to provide supplemental services and did not provide any data on 
numbers of students. The estimates of the number of students who received services are based on the 48 percent 
of districts with schools required to offer supplemental services and provided supplemental services to students in 
identified Title I schools; this is also true of the estimated number of low-income students eligible to receive 
supplemental services in districts offering these services (hence the number of students receiving services is a 
more accurate estimate). 
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¢ Another 14,000 students (2 percent) in districts that offered supplemental services 
received services from providers that were not approved by the state (e.g., delays in 
states identifying approved providers or a lack of approved providers in the area 
encouraged some districts to begin providing some sort of supplemental services to 
help students before approved providers became available). 


Again, there were significant differences at the district level in the number of e/igible students by 
district size and location, and by district size in the number of students receiving supplemental 
services. So, for example, the average number of eligible students in suburban districts that 
provided supplemental services was around 1,500, but in urban districts the average was around 
7,200 students. The average number of students that received supplemental services in medium 
districts was 94, whereas the average number of students was 535 in very large districts. At the 
school level there was also some variation (see Exhibit 32). 


The number of Title I students that received supplemental services varied 
considerably across schools. The average number was higher in schools with higher poverty 
levels. More than a third (37 percent) of continuously identified Title I schools had none or 
fewer than 10 students receiving services as of spring 2003 (see Exhibit 32). However, 16 
percent had more than 100 of their students receiving services. As a result, the median number 
of students who received supplemental services in Title I schools that continued to be identified 
in 2002-03 (in districts that provided services) was 24 whereas the average was 64. 


Exhibit 32 


Distribution of Continuously Identified Schools That Offered 
Supplemental Services by Number of Participating Students 


Number of students who received Title | Percentage of schools where 
supplemental services: supplemental services offered 
None 22 
1 to 9 students 15 
10 to 99 students 47 
100 or more students 16 


Exhibit reads: Among Title I schools in districts that offered supplemental services, 22 
percent had no students receive supplemental services in 2002-03. 


Note: See the appendix for sample sizes and additional statistical information. 


Source: TASSIE principal survey. 


Several considerations influenced parents’ decisions when offered supplemental 
services for their children, and transportation issues were a deciding factor for many 
parents. Parents often chose providers more for logistics and convenience than for educational 
concerns as revealed in parent focus groups. How far children had to travel for services was a 
critical issue for parents. In a small rural district, for example, parents did not want to bus their 
children 20 miles after school to receive services, particularly because the identified school was 
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offering programs similar to supplemental services taught by teachers they knew. (Alternatively, 
parents in a large district indicated that they would have definitely taken advantage of 
supplemental services if they were available because they felt that the after-school programs on- 
site were more like homework clubs than targeted intervention.) A large urban district tried to 
minimize travel logistics by recommending providers that were nearest identified schools. In 
addition, parents of eligible students interviewed generally said they liked and trusted their 
children’s schools and teachers, even though the schools were identified for improvement. 
Parents often based their judgment of a school on the accessibility and personality of the teacher 
(e.g., if he or she was caring or not) and the school environment. In some cases, inadequate 
parent information about supplemental services, as discussed later in this section, may have 
affected parents’ decisions as well. 


Parent focus group data were reflected in the results of a 2003 national poll of the public’s 
attitudes toward public schools, where 54 percent of public school parents indicated they would 
have preferred tutoring provided by teachers in their child’s school, and 42 percent would have 
preferred tutoring provided by an outside agency if their child had been in a school identified for 
improvement (four percent did not know) (Lowell and Gallup 2003). Case studies of 15 districts 
conducted in a 2002-03 study of the implementation of NCLB reported similar findings (Center 
on Education Policy 2003). Officials in several of these case study districts attributed low 
participation in tutoring services to “the limited number of approved providers available in some 
areas, the late start for initiating this requirement, difficulties in managing and administering the 
program, and the need to change families’ concepts of ‘school’ to encompass a longer learning 
day in a variety of settings” (p. 5). 


Among districts that provided supplemental services, 62 percent made these services 
available to all students from low-income families because demand did not exceed Title I 
funding requirements. Case study data from 2002-03 suggest that districts were taking various 
approaches to determine how to prioritize services when demand exceeded resources.” A large 
urban case study district prioritized services to the lowest achieving low-income students 
because demand exceeded resources, but this was not the situation in other districts. Before 
determining whether services needed to be targeted or prioritized, several districts in a separate 
set of case studies considered the number of students attending identified schools and the amount 
of Title I dollars available to fund supplemental services. Other districts determined whether or 
not it was necessary to assign priority for services on the basis of the initial response they 
received from the letters sent to parents of eligible students regarding the availability of 
supplemental services (ED 2004b). 


As noted above, more than half (52 percent) of districts required to offer supplemental 
services (primarily small, rural, and medium- to high-poverty districts) were not implementing 
supplemental services in 2002-03. The most frequent reason districts cited for not having 
provided services was that no parents had signed up for services (47 percent), but a quarter of the 
districts (24 percent) were still notifying parents and setting up services, and 17 percent had not 


> According to ED’s non-regulatory guidance, all students from low-income families who attend Title I schools 
identified for the second year or more of school improvement are eligible to receive supplemental services. 
However, if available district funds are insufficient to provide supplemental educational services to eligible 
students, the school district must give priority to the lowest-achieving students (ED 2003b). 
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yet received a list of providers from the state.”° According to focus groups with parents, 
awareness of options appeared to be only one factor that influenced their choice of services. 
Few parents requested supplemental services for reasons that included preexisting after-school 
programs or providers located far from the school. 


Fewer than half the principals of schools required to offer supplemental services 
reported that they had eligible students. In Title I schools that continued to be identified in 
2002-03 (i.e., identified for at least two years), 44 percent reported that they had students eligible 
to receive supplemental services. Because NCLB requires that students in low-income families 
in Title I schools in their second year of improvement be offered supplemental services, these 
numbers suggest that principals did not understand this requirement. Other explanations were 
suggested by the case study sites, including the transitional nature of AYP definitions. In two 
case study states, districts were told by their states to delay offering supplemental services in 
schools that might be identified for two consecutive years until new AYP definitions were in 
place. Both Michigan and Washington were changing their AYP definitions, and thus defining a 
school’s status during the 2002-03 school year was subject to considerable confusion. Asa 
result, the number of schools that offered supplemental services in these states may have been 
less than expected, and principals’ knowledge about the requirement to offer supplemental 
services to their students may have been compromised. 


Lack of providers in the area was the most significant challenge to districts in 2002-03. 
Districts with identified schools that provided services reported that the most significant 
challenge they faced in implementing supplemental services in 2002-03 was the lack of providers 
in the area (48 percent). Sixty-one percent of districts cited serious challenges not listed on the 
survey (see Exhibit 33). These challenges included not having been provided a list of service 
providers by the state, services offered by providers that did not match local needs (e.g., services 
available online but parents had no computers, no services for grades one to four), the inadequate 
quality of providers available, and service contract issues such as responsibility for students with 
poor attendance—reasons similar to those reported by districts that had not yet begun providing 
services. The survey data reflected the experiences of case study districts and the issues 
considered by parents in selecting services as described in parent focus groups. 


°° Districts may be granted a waiver to not provide supplemental services if no approved providers make services 
available in the general geographic location served by the district or via distance learning and if the district 
provides evidence that it cannot provide services (ED 2003b). But case study data suggest that parents may 
define “a reasonable distance” within or outside the boundaries of a district differently than the district does. 
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Exhibit 33 


Challenges Faced by Districts That Implemented Supplemental Services, 
Among Districts That Provided Services 


Great Moderate Small 
Type of challenge extent extent extent Not at all 
Lack of providers in the area 48 6 6 41 
Lack of an established reputation for 18 27 24 30 
providers with parents 
Competition from existing after-school 16 8 33 44 


programs 


Inadequacy of an amount equal to 20% of 
the district's Title | allocation for meeting all 
requests for services or providing 6 3 11 81 
transportation to all students who request a 
transfer to alternate schools 


Inadequate time for parents to learn about 3 13 29 62 
supplemental services 
Inadequate information for parents about 

; 1 8 20 71 
supplemental services 
Other 61 19 7 13 


Exhibit reads: Among districts that provided supplemental services in 2002-03, 48 percent responded that a lack of 
providers in the area was a challenge to a great extent to implementing supplemental services, 6 percent responded 
that such a lack was a challenge to a moderate extent, 6 percent responded that it was a challenge to a small extent, 
and 41 percent responded that it was not a challenge at all. 


Note: Totals may not add to 100 percent because of rounding. See the appendix for sample sizes and additional 
statistical information. 


Source: TASSIE district survey. 


Communication With Parents 


A majority of districts with identified Title I schools missed the deadline for notifying 
parents of their public school choice option before the beginning of the 2002-03 school year. 
NCLB calls for districts to notify parents about eligibility for public school choice before the 
beginning of the school year (the first day of school), but some leeway was granted for the 2002- 
03 school year (i.e., by the beginning of the 2002-03 school year) as outlined in the U.S. 
Department of Education's non-regulatory guidance (ED 2002). As shown in Exhibit 34, 

45 percent of districts with identified Title I schools that offered choice notified parents about 
their option for Title I public school choice before the beginning of the 2002-03 school year; on 
average seven weeks before school began. Notification continued through the school year (on 
average 10 weeks into the school year). 
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Exhibit 34 


Timing of Notification to Parents of Students in Identified Title | Schools 
About Eligibility for School Choice, Among Districts That Offered Choice 


Average 
Percent of number of 
When parents were notified districts weeks 
Before the beginning of the 2002-03 school year 45 7 
At the beginning of the 2002-03 school year 23 NA 
After the beginning of the 2002-3 school year 42 10 


Exhibit reads: Among districts with identified schools that offered choice options, 45 percent notified 
parents of children in identified Title I schools about their eligibility for public school choice before 
the beginning of the 2002-03 school year. The average timing of notification was seven weeks before 
the school year began. 


Note: Percentages add to more than 100 percent because some districts notified parents more than 
once. See the appendix for sample sizes and additional statistical information. 


Source: TASSIE district survey. 


Both districts and schools had taken part in the notification process regarding public 
school choice and supplemental services, with most having relied on written 
communication. Almost all districts sent written notices to parents; just slightly fewer than half 
held parent meetings (see Exhibit 35). Twelve percent of districts surveyed reported that they 
used other means (other than those listed in Exhibit 35) to communicate with parents about their 
school choice options, including local newspapers, local cable television, the district Web site, 
newsletters, computerized phone messages to parents, parent and community or church network 
group phone hot lines, and the district welcome or enrollment center. Many districts (87 
percent) used more than one method to communicate with parents regarding school choice and a 
little more than half (57 percent) of districts had used more than one method to communicate 
with parents regarding supplemental services. Case study data also suggest that more than one 
form of communication was often used in 2002-03. 


About a third of continuously identified schools (32 percent) reported that the district alone 
handled communication regarding school choice, and a similar percentage reported that the 
district alone handled communication regarding supplemental services. However, schools were 
involved in other activities such as collecting information on eligible students and responding to 
parents’ questions about the information sent by the district or to questions about the types of 
services that were going to be offered. The role the schools played in communicating with 
parents was critical as seen in some case study districts. For example, in a very large urban case 
study district, parents felt a close connection only to school staff and therefore looked to them 
for information about the status of the school and their school choice options. In a separate set 
of case studies on supplemental services parents reported frequently relying on their children’s 
teachers and schools to help them make decisions about service providers (ED 2004b). 
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Exhibit 35 


Methods Used by Districts to Communicate School Choice and Supplemental Services 
Options to Parents, Among Districts With Identified Schools and That Offered 
Title | Choice and Supplemental Services 


Percent of districts 
with identified Title | schools 


About About 
school supplemental 
Communication method choice services 
Written notification sent home to parents about school 
‘ i , 89 99 
choice options or supplemental services available to them 
Parent meetings to publicize and discuss school choice 45 43 


options or supplemental services 


Discussions with parents held by classroom teachers, 
principals, and other school staff about supplemental NA 50 
services options 


Individual meetings with interested parents NA 40 


Public service announcements 22 32 


Enrollment fairs or other events where parents of 
students in schools identified for improvement can learn 


about alternate schools or about providers and the : 13 
services they provide 

Open houses in alternate schools for parents of students 

: : ve : 6 NA 
in schools identified for improvement 

Other methods 12 6 


Exhibit reads: Among districts with identified Title I schools, 89 percent that offered choice used written 
notification to communicate with parents about the school choice options available to them in 2002-03, and 99 
percent that offered supplemental services used written notification to communicate with the parents of eligible 
students about supplemental services. 


Note: See the appendix for sample sizes and additional statistical information. 


Source: TASSIE district survey. 


Parents did not necessarily understand the information contained in notification 
materials. Case study data indicated that both language barriers and confusing terminology 
contributed to the lack of understanding. In two urban districts, the need to translate letters into 
multiple languages to communicate with non-English speaking parents was a problem. One of 
these districts had the resources to deal with this issue, hiring bilingual staff and translating 
information for parents into 10 languages. The other district reported inadequate staff and funds 
to address this issue, and letters informing parents of students in identified schools of the option 
to transfer their children to non-identified schools were sent out only in English, even though the 
district serves a population speaking 36 languages. Sometimes confusion stemmed from the 
presence of state accountability initiatives that operated in conjunction with their Title I 
accountability systems under NCLB. In Arizona, for example, the different ratings given to a 
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school under the state initiative (AZ LEARNS) versus the state’s accountability system for 
NCLB has caused considerable confusion among parents because the various labels used sound 
contradictory. The new state labeling process had in essence created a dual accountability 
system because a school could receive a “maintaining” or “improving” label from the state but 
still be considered in school improvement under NCLB. 


Some districts downplayed the information provided to parents on Title I choice, 
whereas others had taken a number of steps to inform parents about their choice options. 
In some cases, district notices focused on the benefits offered at identified schools and 
downplayed the information on choice (see Exhibit 36). 


Exhibit 36 
Examples of Parent Information on Public School Choice 


In one case study district, the letters sent to parents in October 2002 never directly used the term 
“identified school.” Instead, the district notified parents that their school had been “designated to 
receive Title | School Improvement Funds this year...to be used in our intensive school improvement 
activities” and that the school qualified for these funds because of “past achievement scores.” It was 
not until the third paragraph that parents were told that the district “offers you choices in which school 
your child attends” and that the school would be “delighted to work with you regarding the options that 
exist.” As one respondent noted, the letters read as though the school had just won something. 


At an identified elementary school in another district, the principal held parent meetings to discuss the 
school’s status. She stressed the additional resources that had become available as a result of the 
school having moved up to the next level of identification under the state’s system—‘“I made the status 
positive.” A combination of state and federal funds supported the addition of a social worker, a part- 
time nurse and curriculum specialist, two additional teachers, a vice-principal, more resources for 
professional development and materials, and a parent center. In this case, only five parents chose 
another school, and parent involvement generally increased. 


Parents in case study districts reported receiving varying amounts of information about 
their supplemental service provider options. Some parents received nothing but a letter from the 
district informing them of the supplemental services provisions of NCLB. Other parents had 
been invited to district- or school-sponsored meetings held to clarify their options for 
supplemental services. Some parents indicated that they found out about providers and services 
through the news media. In other cases, parents indicated that they had received very little 
information on the topic, if any, in 2002-03. 


Some districts went beyond the basic requirements in the law regarding parent notification 
to generate parent interest in supplemental services. For example, a district in a separate set of 
case studies hosted vendor fairs at the identified schools, used Title I district parent involvement 
funds to pay teachers to call parents to encourage them to apply for tutoring services, and 
encouraged principals and other school staff to conduct home visits to invite parents to take 
advantage of the supplemental services opportunity. Nevertheless, despite its carefully planned 
efforts to encourage parent participation, the district did not succeed in filling all the slots it 
expected to fund for supplemental services (ED 2004b). 
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Roughly a fifth of districts with eligible students in 2002-03 had not yet communicated 
with parents regarding their right to receive supplemental services. Twenty-one percent of 
districts with eligible students had not yet communicated with parents as of spring 2003 because 
the district had not yet begun to provide supplemental services. (Reasons for the delays in 
service delivery were discussed earlier in this chapter.) 
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IV Corrective Actions 


Planning and support are intermediate steps that districts take with low-performing 
schools, along with offering parents the option of school choice and supplemental services. If 
schools do not make AYP for four years, districts are required to impose more intense 
interventions. The combination of support to identified schools and consequences for continued 
poor performance are key elements of NCLB and state accountability systems, and are intended 
to drive school improvement by creating incentives for educators to improve their practice. 


The data in 2002-03 suggest that the impacts of corrective actions had not been broad. The 
data also indicate that the pattern of interventions did not change under NCLB 
requirements—the pattern of interventions most frequently employed by districts with identified 
schools was similar to that in 2001-02 before the implementation of NCLB. 


Corrective Actions 


If Title I schools continue not to make AYP for two years after initial identification (i.e., do 
not make AYP for four years), NCLB requires districts to take a least one of a series of six 
corrective actions defined in the law and consistent with state law, Section 1116(b)(7): (1) 
requiring the implementation of a new research-based curriculum or instructional program, (2) 
significantly decreasing the management authority level at the school level, (3) appointing an 
outside expert to advise the school, (4) extending the school day or year, (5) restructuring the 
internal organization of a school, or (6) replacing school staff who are relevant to the failure to 
make AYP. 


Because only a few districts reported that they had schools identified for 
improvement for three years or more in 2002-03, few were required to take the corrective 
actions specified under NCLB. Approximately 400 districts, or just 4 percent of all districts 
nationwide, reported that they had Title I schools identified for improvement for three years or 
more. However, because of rules under IASA and transition rules under NCLB regarding school 
status in 2002-03, as discussed in Chapter I, not all schools identified for improvement for three 
years in 2002-03 were in corrective action status. Thus the number of districts required to take 
corrective action may have been even lower in 2002-03. At the same time, some districts were 
unable to report how long some of their schools had been identified for improvement, meaning 
that the number could also be underestimated. About a third of the districts with identified 
schools in 2002-03 were unable to provide information about how long those schools had been 
identified for improvement. 


The corrective actions most frequently taken in 2002-03 could be characterized as 
assistance rather than interventions—a pattern similar to that in 2001-02 before the 
implementation of NCLB. Corrective actions taken ranged from those that overlap with 
assistance and support (e.g., requiring a new research-based curriculum) to those that are clearly 
interventions (e.g., replacing school staff). The expectation that the pattern of interventions 
would change with the mandates of NCLB was not supported by this study’s 2002-03 data. 
Under IASA, districts were not required to take corrective actions if their states did not yet have 
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an assessment system approved under Title I, and only 21 states had these final assessment 
systems in place as of October 2002. Because NCLB requires districts to impose corrective 
actions no matter what the approval status of their state’s assessment and accountability systems, 
it was predicted that these stronger consequences would occur more frequently under NCLB. 
Given the small number of districts with schools identified for improvement for three years or 
more (or subject to corrective actions) in 2002-03, this pattern might change in the future. 


Exhibit 37 compares interventions taken by districts with schools identified for different 
numbers of years to assess whether or not districts with schools identified for a longer period of 
time (three or more years in this case) were taking different types of corrective actions compared 
with districts that had schools identified for only one or two years. The data indicated that the 
most common corrective actions taken in 2002-03 were requiring implementation of a new 
research-based program, appointing an outside expert, and extending the school day or year 
regardless of the number of years of identification. Far fewer districts applied interventions such 
as replacing the school staff who were relevant to the failure to make AYP. 


In 2002-03, districts with schools identified for improvement for three years or more 
were no more likely to take corrective actions with their identified schools than districts 
that had schools identified for only one or two years. Compared with districts that only had 
schools identified for one or two years, districts with schools identified for three years or more 
were equally likely to require curriculum adoption, appoint an outside expert, decrease school 
management authority, or take any of the other corrective actions in NCLB. These data are 
consistent with district assistance findings regarding targeted support for school improvement 
(see Exhibit 18 in Chapter IT) that showed that districts tended to provide some form of support 
(e.g., revising the school improvement plan, on-site assistance) to all their schools, particularly 
identification of research-based strategies. In addition, a majority of identified schools had 
adopted new curricula. In this context, it was not surprising that actions taken with schools 
identified for three or more years were similar to those taken with other identified schools. 


¢ Some districts required schools in corrective action to undertake more intensive and 
systematic planning activities for instructional improvement, as well as increased 
monitoring of school activities (see Exhibit 38). 


¢ Insome districts, actions such as replacing the principal—a common occurrence in 
urban districts—were not called corrective actions, thereby mitigating the roles of some 
corrective actions as incentives to improve school performance. 


¢ At the school level, some principals did not emphasize corrective actions because they 
believed it was more beneficial to stress improvements. For example, a principal in an 
identified school indicated she was more focused on making improvements by bringing 
school staff along through professional development and participation in the school 
reform model rather than emphasizing potential corrective actions. A coach at the 
school commented: “We hear about corrective action, but people are working too hard, 
we can’t focus on it. We’re trying to be more effective.” A principal in another 
elementary school stated: “I try to keep these things [AYP definitions, program 
improvement status] away from teachers. The hammer is being held over my head, so I 
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don’t need to hold it over my teachers’ heads. They are working very hard. I try to 
keep them insulated. I tell them not to worry about it.” 


Exhibit 37 


District Actions Taken With Schools Identified for Improvement in 2002-03, 
by Number of Years of Identification 


school 


Percent of 
Percent of districts with Percent of all 
districts with schools districts with 
schools identified for 3 schools 
identified 1 or 2 or more years identified for 3 
Interventions years only only or more years 
Requiring the implementation of a new 
research-based curriculum/instructional 53 56 55 
program 
Significantly decreasing management 7 14 15 
authority level at the school level 
Appointing an outside expert to advise the 45 30 AY 
school 
Extending the school day or year 34 61 54 
Restructuring the internal organization of the 
20 8 8 
school 
Replacing school staff who are relevant to 4 9 5 
the failure to make AYP 
Reassigning or demoting the principal 17 13 
Replacing all or most of the school staff 2 
Reopening the school as a public charter 1 0 0 


Entering into a contract with a private 
management company to operate the school 


1 


1 


1 


Having the state take over the school 


1 


1 


1 


Exhibit reads: Among districts with schools identified for one or two years only, 53 percent required identified schools 
to implement a new research-based curriculum or instructional program, whereas 56 percent of districts with schools 
identified for three or more years only (and not one or two years) required these schools to take this action. Among all 
districts with schools identified for three or more years, 55 percent required these schools to take this action. 


Note: See the appendix for sample sizes and additional statistical information. 


Source: TASSIE district survey. 
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Exhibit 38 
Corrective Actions Taken With Identified Title | Schools 


A large urban district focused attention on seven Title | schools identified by the state as “intensive 
schools” and identified for corrective action under Title |. In these “intensive schools,” fewer than 25 
percent of students scored satisfactory or above on the state assessment during the 2001-02 school year 
in at least two of the three content areas of mathematics, reading, and science, nor did the school make 
AYP for two years in a row in the same content area in which performance had been low. The state 
provided these schools with grants from its school improvement reserve funds~’ ($95,000 per school 
initially), and the district required these schools to adopt research-based instructional approaches and 
receive advice from outside experts (in this case, district specialists), along with intensified district 
monitoring of the use of school budgets and classroom activities. In total, the district required these 
schools to carry out both school improvement activities and corrective actions: (1) revise their school 
improvement plans to incorporate research-based “high-yield strategies” suggested by district staff (e.g., 
specific instructional strategies, ongoing collaboration and support among staff, use of data or diagnostic 
assessments to guide instruction); (2) write a grant application for the state school improvement reserve 
funds that reflected the goals and activities in their school plans; (3) attend meetings twice a month with 
the core district academic staff, including Title | staff, and once a month with the other intensive schools; 
(4) develop indicators of student progress against goals in the school improvement plans; and (5) have 
the principals and assistant principals conduct walk-throughs of classrooms to observe implementation of 
instructional strategies. 


Survey data supplied by principals supported district reports regarding the types of 
corrective actions taken with identified schools. Principals in schools that continued to be 
identified in 2002-03 (identified for at least two years) reported that the three most frequent 
corrective actions taken with their schools were: requiring the implementation of a new research- 
based curriculum or instructional program (82 percent); extending the school day or year (56 
percent); and appointing an outside expert to advise the school (51 percent). Except with regard 
to the implementation of a new research-based curriculum or instructional program, there were 
no significant differences in the interventions imposed on schools identified for two years or 
more, compared with schools identified for at least three years. 


Even though much of the curriculum adoption carried out in districts applied to all schools 
as described in Chapter II, some identified schools were also required to implement specific 
curricula or instructional practices to address low performance as seen in case study districts (see 
Exhibit 39). 


*7 Section 1003 of NCLB requires that each state reserve 2 percent of the amount the state receives under Subpart 2, 
Part A, of Title I funds for fiscal years 2002 and 2003, and 4 percent of the amount received for fiscal years 2004 
through 2007 to carry out the state education agency’s school improvement responsibilities (e.g., establishing a 
statewide system of technical support for districts with identified schools). 
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Exhibit 39 
Example of Identified Schools Required to Implement New Instructional Approach 


In a very large urban district, identified schools that had not made AYP for three years and might move 
up to the next level of interventions under the state accountability system were required to implement 
one of two school reform models that specified a particular instructional approach—Success for All or 
Direct Instruction; the schools were also required to implement the district’s new reading and language 
arts curriculum. To assist with the implementation of these programs, staff received five days of 
professional development at the beginning of the 2003-04 school year, displacing the three days of 
preservice professional development that teachers normally received. The professional development 
was customized to address the problems related to reading achievement at each school. During the 
school year, staff received four to five more training sessions at their schools through their school- 
based facilitators. An outside consultant made visits on a periodic basis to monitor implementation and 
answer questions. In this state, as part of the state’s accountability system, schools that do not make 
AYP for three or more years must also receive the services of a full-time distinguished educator. 


Due to the transition rules described in Chapter I, for 2002-03, there was no requirement 
that schools be identified for restructuring status. At the same time, some districts took actions 
with identified schools that are included in the five interventions outlined in NCLB for school 
restructuring: (1) replacing all or most of the school staff, (2) reopening the school as a public 
charter school, (3) entering into a contract with a private management company to operate the 
school, (4) having the state take over the school, or (5) other major restructuring initiative. 
Districts that took these actions most frequently reported replacing all or most of the staff (see 
Exhibit 37). 


Change Over Time 


The most frequent corrective action taken in both 2001-02 and 2002-03 with Title I 
schools identified for improvement was the requirement to implement a new curriculum or 
instructional program. The next most frequent actions were appointing an outside expert to 
advise the school and extending the school day or year. As noted above, the expectation was 
that the pattern of interventions would change with the requirements of NCLB. In general, the 
expectation was that, because NCLB now required districts to take specific corrective actions 
with schools that did not make AYP for two years after being identified, changes would occur in 
the types of interventions taken with these schools. To test this hypothesis, the corrective actions 
taken by districts that had identified schools in 2001-02 and 2002-03 were compared. Exhibit 40 
illustrates the actions taken by districts that had identified schools in both 2001-02 and 2002-03. 
None of the differences across years were statistically significant. 
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Exhibit 40 


District Actions Taken in 2001-02 and 2002-03 With Schools Identified for 
Improvement, Among Districts That Had Identified Schools in Both Years 


Interventions 2001-02 2002-03 

Requiring the implementation of a new research-based 46 57 
curriculum/instructional program 
Significantly decreasing management authority level at 17 42 
the school level 
Appointing an outside expert to advise the school 35 49 
Extending the school day or year 40 40 
Reassigning or demoting the principal 6 12 
Replacing all or most of the school staff 4 1 
Entering into a contract with a private management 

<1 <1 
company to operate the school 
Having the state take over the school <1 <1 


Exhibit reads: Among districts that had schools identified in both years, 46 percent reported that 
they had required identified schools to implement a new research-based curriculum or instructional 
program in 2001-02, and 57 percent reported that they imposed this requirement on identified 
schools in 2002-03. 


Note: There were no statistically significant differences across years among these interventions. 
See the appendix for sample sizes and additional statistical information. 


Source: TASSIE district survey. 


Several factors may have contributed to the lack of shifts in the types of corrective actions 
taken by districts as illustrated by case study data. As noted earlier, because some districts were 
in states that were revising their AYP definitions, they had not imposed the corrective actions on 
schools that they might otherwise have done. Case study districts used a combination of 
corrective actions and additional resources or technical assistance to improve all low-performing 
schools. Staff interviews suggest that a focus on interventions without assistance could generate 
negative responses from school staff. Together, assistance and corrective actions encouraged 
staff to reexamine current practices, to place more focus on instruction, and to place a greater 
urgency on making reforms. 


Nonetheless, compared with 2001-02, there was a significant increase in the 
percentage of districts with identified schools reporting that they were increasing their 
monitoring and district oversight of identified schools, as illustrated in the case study 
examples provided above and in comparisons of survey data. In 2001-02, 23 percent of districts 
with identified schools (regardless of the number of years of identification) reported increasing 
monitoring and district oversight of identified schools; in 2002-03, 34 percent of districts with 
had identified schools in both years reported taking this action with their identified schools. In 
addition, a higher percentage of districts with schools identified for three years or more in both 
2001-02 and 2002-03 reported that these schools had been given less discretion over school-level 
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spending and had less autonomy in selecting school improvement strategies compared with other 
schools identified for improvement (see Exhibit 41). 


Exhibit 41 


District Actions Taken in 2001-02 and 2002-03 With Schools Identified for 
Improvement for Three or More Years, Among Districts That Had 
Identified Schools in Both Years 


Percent of districts 
Compared with other schools identified for reporing fo great 
. : ee ; extent 
improvement, schools identified for improvement for 
three or more years... 2001-02 2002-03 
Have less discretion over school-level spending 4 24 
Have less autonomy in selecting school improvement 4 29 
strategies 


Exhibit reads: Among districts with corrective action schools or schools identified for improvement 
for three or more years responding in both years, 4 percent of districts reported in 2001-02 that those 
schools had less discretion over school-level spending to a great extent compared with other schools 
identified for improvement. By comparison, 24 percent of districts in 2002-03 reported these 
actions. 


Note: Differences were statistically significant for both actions. See the appendix for sample sizes 
and additional statistical information. 


Source: TASSIE district survey. 
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V Conclusions 


Findings of the Evaluation of Title I Accountability Systems and School Improvement Efforts 
(TASSIE) capture the first full year of implementation of the accountability provisions of No 
Child Left Behind. They show that states, districts, and schools were making progress in meeting 
NCLB requirements. States and districts were taking steps to create new accountability systems, 
to measure and communicate about schools’ progress, and to devise strategies to improve schools. 
Schools created plans and adopted new instructional programs. However, progress was uneven, 
and a big gap remained between the existing systems of accountability and the vision embodied in 
NCLB to create systems that support all schools and students to attain high standards. 


Data from 2002-03 offer a first glimpse into the influence of NCLB on improving school 
performance. Identified schools and districts in 2002-03 were the first to be subject to many 
NLCB accountability requirements, and 2002-03 was the first year many related NCLB 
requirements at the state and district levels were in effect. Declines by about a third in both the 
number of identified schools (from 9,000 to 6,000) and the number of districts with at least one 
such school (from 2,900 to 1,900) from 2001-02 to 2002-03 directly affected which schools and 
districts were subject to NCLB accountability requirements. Because of transitional policies that 
applied for only 2002-03 and new definitions of AYP under NCLB to be implemented in 2003- 
04, inferences about trends in numbers of schools and districts identified for improvement 
should be made with caution. Similarly, case study data suggest that some confusion about how 
NCLB accountability provisions applied may have hampered full implementation in 2002-03. 
With the implementation of new AYP definitions under NCLB following 2002-03 and growing 
understanding of NCLB accountability provisions over time, the effect of NCLB likely will 
become more visible. 


Nevertheless, educators at all levels of the education system were responding to NCLB’s 
accountability requirements. For each major provision, many policymakers and practitioners 
took action to build accountability systems that complied with NCLB and made sense in their 
own jurisdictions. At the same time, findings from 2002-03 show that some substantial 
improvements in implementation were needed in order for states, districts, and schools to be 
fully implementing NCLB accountability requirements. Highlights of 2002-03 included: 


* Most states and districts reported on student and school assessment results and made 
the information available to the public through multiple channels, but not all of NCLB 
reporting requirements had been implemented. 


¢ Many states and districts provided technical assistance, additional staff and 
professional development to schools identified for improvement. Yet two-thirds of 
continuously identified schools reported no access to certain key forms of state or 
district on-site assistance (e.g., full-time staff to support teacher development), either 
because they were in districts and states that did not provide that assistance or because 
states or districts could not serve all of their identified schools. 


tg, 


¢ To improve their students’ performance, many identified schools gave increased 
attention to achievement results, adopted new curricula, used school reform models or 
added new supplemental instructional programs. At the same time, the adoption of new 
curricula or school reform models was often not accompanied by related professional 
development and other supports. 


¢ Some districts and schools had organizational structures in place that could provide and 
take advantage of assistance; but many—especially small and rural districts—did not. 


¢ An increasing percentage of districts offered students in identified schools the choice of 
moving to other public schools that were not identified. However, a third of the 
districts with schools identified for improvement did not provide public school choice 
in 2002-03. 


¢ Some districts began to provide access to supplemental educational services, but half of 
the districts with schools identified for two or more years did not offer supplemental 
services. 


¢ Even where public school choice and supplemental services were offered, only a small 
fraction of parents of eligible students exercised these options. 


The study’s survey and interview data suggest broad support for the goals of NCLB and 
serious attempts to implement the new accountability provisions. Moving forward, however, 
states and districts faced significant challenges in 2002-03 in meeting the law’s goal of changing 
instructional practices in ways that will ensure that no child is left behind. 


Among these challenges was a shortage of capacity in the broadest sense of the word: 
enough people with the knowledge and skills to serve as distinguished educators or school 
support team members or mentors to help schools improve; enough supplemental service 
providers to meet the need, especially in rural schools; enough translators and staff to adequately 
inform all parents of their options. Staff in districts with large numbers of identified schools 
were already stretched to the limits to provide sufficient help to all eligible schools, and smaller 
districts lacked the professional staff to provide the help required. As more Title I schools are 
identified for improvement, this challenge will likely increase both for districts and state 
departments of education whose staff are similarly stretched thin. District and school staff also 
need access to greater expertise in curriculum, instruction, assessment and professional 
development, both within and outside the district. Such expertise will be especially important if 
district and school staff are to move beyond support for planning and alignment to help schools 
make the classroom-level changes that will help students reach proficiency. 


Although the 2002-03 school year was the first full year of implementation of NCLB, the 
general lag between the passage of a new law and the time when schools and their communities 
grasp its implications suggests that the 2003-04 school year, the final year of this study, will 
show how implementation is changing over time. 
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Appendix 
Method Notes 


Sampling Information 


The evaluation consists of five related, longitudinal components: 


A survey of Title I administrators in a nationally representative sample of 
approximately 1,300 districts that receive Title I funds. The universe of eligible 
districts was developed using information from the 1999 Common Core of Data (CCD) 
and the 2000 Quality Education Data (QED) database. Districts were stratified 
according to size (enrollment), degree of poverty (based on the percentage of children 
living in poverty within each district), and geographic region.”** The stratification by 
geographic region was done to facilitate selection of an oversample from three states 
that are the focus of the analysis of school performance. Districts were selected using a 
simple random sample without replacement within each stratum. All very large 
districts were sampled; approximately equal numbers of districts were selected from 
the other size strata. Each poverty stratum includes approximately one-third of all 
children in the sampling frame. The key respondent is the district Title I director. 
Survey topics for 2002-03 included state and local accountability systems, school and 
district identification, district support for school improvement, public reporting, public 
school choice, supplemental services, and interventions taken with identified schools. 
Response rates in 2002-03 were 89 percent. 


A survey of principals in a national sample of 739 Title I schools identified as in 
need of improvement in 2001-02. The sampling frame for schools in need of 
improvement in 2001-02 was developed in a two-stage process. First, lists of the 
schools in each of the sampled districts were developed from the 1999 CCD and the 
2000 QED databases. Schools eligible for the sample were classified as regular, but 
not charter, in the CCD and served a grade range that could be classified as elementary, 
middle, or high school. In the second stage, sampled districts were asked to provide a 
list of current Title I schools in need of improvement at the time of the request in fall 
2001. In states where Title I schools in need of improvement are identified by the state 
education agency (SEA), the list of schools identified on the basis of 2000-01 
assessment data was requested. Only Title I schools identified for improvement in 
reading, mathematics, or both subjects were included in the sampling frame. The 
sampling frame of the resulting 4,054 Title I schools in need of improvement was 
stratified by district size, school level (elementary, middle, or high), poverty level, and 
geography. The sample was a nationally representative sample of Title I schools 
identified for improvement in 2001-02, but because schools continue to be surveyed for 
three years even if they move out of improvement, the sample is not a nationally 
representative sample over time (i.e., in 2002-03 and 2003-04 or the second and third 
years of data collection). Survey topics for 2002-03 included accountability criteria, 
public school choice, supplemental educational services, school improvement 


*8 See Exhibit 2 for definitions of size and poverty strata. 
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activities, and interventions taken with the school. Response rates in 2002-03 were 85 
percent. 


¢ Case studies of 20 schools identified for improvement under Title I in 15 districts 
in five states. Case study schools were selected through a multiple-stage process in 
which states, then districts within those states, and then schools within those districts 
were chosen. Three critical dimensions for state sample selection were identified: state 
AYP definition, alignment of Title I and the general state accountability systems, and 
the state process for identifying schools in need of improvement. States were sorted 
along these three dimensions and selected through an iterative process to represent the 
actual variation. Within states, in consultation with the state Title I director, the largest 
urban districts were targeted as well as one suburban and one rural district. Within 
each urban district, two elementary schools identified for improvement were selected in 
consultation with the district Title I coordinator. In the rural and suburban districts, 
one elementary school was chosen (often the only identified school in those districts). 
Respondents include district staff (e.g., Title I director, superintendent, assessment 
director, professional development staff), school staff (e.g., principal, specialists, 
classroom teachers, parent coordinators), and parents. Interviews in 2002-03 were 
conducted with district and school staff and covered topics similar to those on the 
surveys. Parents of students in the identified case study schools were also interviewed 
about public reporting, public school choice, and supplemental services. 


¢ Analyses in three of the case study states of school performance data for all schools 
in the state that serve elementary students. Eligible schools were those classified as 
regular, but not charter, schools, in CCD. The analysis contrasts scores of Title I 
schools in need of improvement, other Title I schools, and non-Title I schools for each 
state. 


¢ State level interviews and analyses of state accountability components: grades and 
subjects in the state assessment systems, elements of the AYP formula, when 
identification information was provided to districts, corrective actions for schools and 
districts, supplemental services, the statewide system of support, and state reporting. 
Key respondents included state Title I directors and accountability staff. 


Weights for Statistical Analyses 


TASSIE district and school samples are stratified, random samples in which the probability 
of selection into the sample varies across strata. To estimate population parameters, the sampled 
districts (or schools) are weighted so that the total of the weights within a stratum equals the 
number of districts (or schools) in that stratum in the sampling frame. 


Survey respondents are a portion of the full sample. To estimate population parameters 
from the survey respondents, the weights assigned to respondents within any stratum were 
modified, as customary in survey analyses, to “absorb” the weights that would otherwise accrue to 
non-responding schools in the stratum. Thus respondents’ weights were adjusted to sum to the 
total number in the stratum. 
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The Year | technical appendix” for TASSIE describes in detail the method used to select 
the samples and the methods used to derive weights for the first year analysis. The analysis for 
Year 2 required two new sets of weights for each survey. The first is the set of weights used to 
summarize Year 2 data. These weights were constructed to estimate parameters for the total 
population of districts (and schools) from respondents to the 2002-03 survey. A second set of 
weights were computed for the longitudinal analyses that examine responses from schools and 
districts that completed the survey in 2001-02 and 2002-03. 


Reconciling Differences in Population Estimates From Different Data Sources 


National estimates for the total number of districts and schools and for the number in certain 
categories, may differ slightly depending on whether the estimate is derived from the TASSIE 
Eligibility file or the TASSIE 2002-03 district survey. For example, an estimate of the total 
number of Title I districts is reported to be 11,230 based on the Eligibility file, while it is 11,197 
(sum of 1,824 and 9,373) based on the survey respondents. An estimated 1,873 districts (16.6 
percent) had at least one identified Title I school in 2002-03 based on the TASSIE Eligibility 
dataset; the estimate is 1,824 districts (16.2 percent) based on the district survey. The 
discrepancies occur because the Eligibility file has complete information on all schools and 
districts in the sample, while the survey data is incomplete due to nonresponse. 


In 2001-02, an estimated 26 percent of districts had at least one identified Title I school 
based on the Eligibility file; the percentage of districts was reported as 21 percent on the basis of 
data gathered from the district survey. The difference in the two estimates for 2001-02 and 2002- 
03 are not statistically significant. In 2001-02, there were an estimated 9,200 identified Title I 
schools based on the TASSIE Eligibility file; the number of identified Title I schools was reported 
as 8,078 on the basis of a weighted analysis of the number of respondents to the principal survey 
(in 2001-02, the survey sample was representative of Title I schools identified for improvement 
based on 2000-01 assessment data). 


The method used to adjust for nonresponse in TASSIE, while customary, assumes that non- 
respondents are distributed in proportions equal to the respondents across other variables, such as 
eligibility for the study. TASSIE is in an unusual position for a survey in having information 
about the eligibility of the districts and schools that did not respond. For future waves of TASSIE 
we intend to modify the weighting procedure to reduce any discrepancies. In the modified 
procedure we will use all known data sources to classify schools prior to adjusting for non- 
response. In essence this extends the concept of “strata” beyond those that were used in the 
original sample selection (1.e., it is a type of post-stratification based in part on external sources of 
data). 


Minimum Sample Size 


We report parameter estimates when they are based on unweighted sample sizes of 20 or 
more. Because of this standard, it is sometimes not possible to report statistics for every cell ina 
cross-tabulation. Interpretations made of the corresponding Chi-square tests and p-values are 


#2 Shields, P. M., Laguarda, K.G., Lash, A., Padilla, C., Winter, N., Woodworth, K., Uperesa, L, Javitz, H. (2004). 
Evaluation of Title I Accountability and School Improvement Efforts (TASSIE): First Year Technical Appendix. 
Menlo Park, Calif.: SRI International. 
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tentative and should be followed up with additional analyses (e.g., of subportions of the tables or 
recoded variables that combine low-frequency response categories) before drawing conclusions. 


Definitions of Scales 


The definitions of the scales used to measure district support and assistance, emphasizing 


related professional development, and teacher collaboration are described below. 


D24. 


D25. 


D36. 


District Support 


Survey items about district support for planning and data use included the following: 


Does the district provide technical assistance with data analysis or planning to identified 
schools? The district assigns staff to work directly with individual schools to: 


a. Review data to be sure identification is valid. 


b. Analyze student achievement data to identify specific academic problems that caused 
the school to be identified. 


c. Identify research-based strategies. 


d. Provide additional data analysis, e.g., additional disaggregation or analysis of 
diagnostic assessments. 


e. Analyze and revise the school’s budget so that school resources are effectively 
allocated. 


f. Review the school’s staffing plan. 
g. Write or revise a school’s improvement plan. 


h. Monitor progress throughout the school year toward goals established in the school 
improvement plan. 


Which of the following topics were addressed in the professional development supported 
by the district in identified schools? 


h. Monitoring individual students’ progress toward learning goals. 
i. Analyzing and interpreting student achievement data. 


Survey items about district support for curriculum alignment included the following: 


Has the district taken any of the following steps to assist schools in ensuring the 
consistency of curriculum and instruction with state or district standards? 


1. Developed local content standards that provide more useful guidance to teachers than 
state content standards. 


2. Published detailed curriculum guides with standards, frameworks, and pacing 
sequences. 


3. Developed classroom-embedded assessments with a standard scoring rubric to monitor 
progress. 


4. Developed model lesson plans based on standards. 


5. Conducted regular checks of standards implementation in classrooms (e.g., by requiring 
lesson plans or students’ work to be submitted, or by conducting walk-throughs). 
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6. Mapped out the alignment of required textbooks and instructional programs to 
standards. 


7. Mapped out the alignment of required textbooks and instructional programs to 
assessments. 


8. Analyzed available student achievement data to identify specific strengths and 
weaknesses related to the attainment of standards. 


D25. Which of the following topics were addressed in the professional development supported 
by the district in identified schools? 


d. Ensuring that curriculum and instruction are consistent with state and/or district content 
standards. 


e. Ensuring that curriculum and instruction are consistent with state and/or district 
assessment. 


Receiving a Minimum Level of Related District Assistance 


District support for implementing a new reading/language arts curriculum was 
constructed by combining responses on six items. A dichotomous scale was created to distinguish 
between those who either agreed or strongly agreed (on average, a score of 4.0 or higher), 
compared to those who endorsed all other response categories (on average). The items included 
the following: 


P39a. District helped to identify reading/language curricula/instructional programs that support 
instruction consistent with standards and assessments. 


P39b. District disseminates useful information about reading/language arts 
curricula/instructional programs. 


P39c. District assisted in selecting a reading/language arts curriculum/instructional program 
appropriate for our school. 


P39e. District is helpful in securing support from publishers/program developers to assist with 
the implementation of our reading/language arts curriculum/instructional program. 


P39f. District provides adequate support for implementation of our reading/language arts 
curriculum/instructional program. 


P39g. District monitors implementation of our school’s reading/language arts 
curriculum/instructional program. 


District support for implementing a new mathematics curriculum was constructed by 
combining responses on six items. A dichotomous scale was created to distinguish between those 
who either agreed or strongly agreed (on average, a score of 4.0 or higher), compared to those 
who endorsed all other response categories (on average). The seven items included: 
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P42a. District helped to identify math curricula/instructional programs that support instruction 
consistent with standards and assessments. 


P42b. District disseminates useful information about math curricula/instructional programs. 


P42c. District assisted in selecting a math curriculum/instructional program appropriate for our 
school. 


P42e. District is helpful in securing support from publishers/program developers to assist with 
the implementation of our math curriculum/instructional program. 


P42f. District provides adequate support for implementation of our math curriculum/ 
instructional program. 


P42g. District monitors implementation of our school’s math curriculum/instructional program. 


District support for the use of a school reform model was constructed by combining 
responses on six items. A dichotomous scale was created to distinguish between those who either 
agreed or strongly agreed (on average, a score of 4.0 or higher), compared to those who endorsed 
all other response categories (on average). The seven items included: 


P36a. District helped to identify school reform models that support instruction consistent with 
standards and assessments. 


P36b. District disseminates useful information about school reform models. 
P36c. District assisted in selecting a school reform model appropriate for our school. 


P36e. District is helpful in securing support from our model’s design team to assist with 
implementation. 


P36f. District provides adequate support for model implementation. 


P36g. District encourages the use of school reform models that focus on instructional 
improvement. 


P36h. District monitors implementation of our reform model. 


Emphasizing Related Professional Development 


Of nine areas of professional development, principals were asked to check the three that 
were the primary areas of emphasis in their schools. If they checked “reading/language arts 
instruction” they were counted as “emphasizing reading/language arts professional development.” 
Analogous definitions were used for “mathematics instruction” and “implementation of a school 
reform model.” This threshold is based on the underlying premise that reports of professional 
development that do not rise to this level are unlikely to be either intensive or frequent enough to 
be considered support for the related improvement activities. 
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Measure of Teacher Collaboration 


Evidence that an identified school was engaged in teacher collaboration as a school 
improvement strategy was defined as "conducting at least one of five teacher collaboration 
activities at least a few times per year.” The five items included: 


P30a. Frequency with which majority of teachers at school participate in teacher work groups to 
analyze samples of student work. 


P30b. Frequency with which majority of teachers at school participate in teacher work groups to 
develop teaching materials or activities for particular classes. 


P30c. Frequency with which majority of teachers at school participate in observations in other 
teachers' classrooms to offer feedback and/or learn ideas (excluding observation for 


purposes of formal evaluation). 


P30d. Frequency with which majority of teachers at school participate in teacher work groups to 
discuss student assessment data to make decisions about instruction. 


P30e. Frequency with which majority of teachers at school participate in in-class coaching or 
mentoring. 


A-7 


